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This paper is one of four individual case study reports presenting the qualitative 
findings from a five-year investigation of the impact of accountability reform on students 
with disabilities in four states, eight districts, and twenty schools. During the late 1980s 
and 1990s, many states passed legislation requiring school systems to be more 
accountable for students’ learning. However, the impetus for the most recent changes in 
state accountability systems originated in state responses to federal concerns about the 
quality and equity of public education. These concerns can be considered as the next 
phase of the standards-based reform movement. 

The standards-based reform movement, which began during the 1970s, has been 
integrated into federal education policy, most notably the last two reauthorizations of the 
Elementary and Secondary Education Act (ESEA), and has become increasingly focused 
on holding schools accountable for the academic performance of all their students. The 
drive for greater accountability and educational equity is embedded in the 2001 
reauthorization of ESEA, known as the No Child Left Behind Act (NCLB). NCLB aims 
to increase student achievement, improve schools, provide parents and the community 
with better information, and close some long-lasting and troubling achievement gaps 
between disadvantaged students and their peers (Cohen, 2002, p. 61). 

The new requirements for standards and assessments are rigorous and largely 
build on the existing Title I requirements promulgated under the Improving America’s 
Schools Act of 1994. For most states however, performance-based accountability 
requires them to chart a course into new and unfamiliar territory. Particularly 
controversial is the requirement that the same annual academic achievement objectives be 
established, met, and reported for specific subgroups, including most students with 
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disabilities, racial/ethnic minority groups, and students with limited English proficiency 
(LEP). 

This case study report presents qualitative data collected over a four-year 
period — 2001-2004 — from three levels of the public education systems, and provides an 
analysis of the impact of accountability reform on students with disabilities in one of our 
participating four states. The following three research questions guided this component of 
the project: 

1 . How do broad education policies that incorporate high-stakes accountability 
include consideration of students with disabilities? 

2. How are students with disabilities affected by education accountability 
reforms? 

3. What impact have educational accountability mechanisms had on students 
with disabilities at the system and individual student level? 

The remainder of this report is divided into five sections. Section I contains 
information regarding the methodology used to collect and analyze the state and local 
data presented in this report. Section II is an overview of the assessment and 
accountability systems in California, including changes resulting from NCLB. In Section 
III we provide demographic information on each of the study sites. In Section IV we 
present our research findings at the state, district, and school level, and in Section V we 
present a discussion of the major findings for each embedded case study and address 
cross-site themes. 
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I. METHODOLOGY 



Identification of Study Sites and Data Sources 

The Education Policy Reform Research Institute (EPRRI), funded by the United 
States Department of Education’s Office of Special Education Programs (OSEP), was 
established in 2000 to investigate and describe the impact of including students with 
disabilities in new state educational accountability systems as required by the 1997 
reauthorization of the Individuals with Disabilities Education Act (IDEA 1997). EPRRI 
researchers adopted an empirical approach to answering the research problem — the case 
study method — and specified a priori the levels of analysis, participants, and questions to 
be pursued. One prerequisite for selection was the requirement that study states have high 
quality data collection systems that could provide disaggregated data for students with 
disabilities. 

Four core study states agreed to participate in the five year study: California, 
Maryland, New York, and Texas. Within each state, EPRRI staff and core state 
representatives identified and secured the participation of two school districts in the 
study. The four selected states varied across several key accountability features, including 
high-stakes versus low-stakes accountability consequences, recentness of reforms, 
stability versus instability of reform efforts, participation of students with disabilities in 
all accountability reports, and use of alternate assessments. In addition, the sites also 
reflected geographic and demographic diversity. It is important to recognize that in 
December 2001 the policy context surrounding this study changed substantially with the 
passage of NCLB and as a result the study sites became more homogeneous in their key 
accountability features. Unfortunately, due to our initial study design we had already 
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selected sites and participants on an a priori basis (versus selecting them purposively and 
serially over time) and it is possible that we limited our opportunities to study relevant 
issues that only became apparent after NCLB and over the course of the study. Despite 
these reservations, we are confident that our analysis of data obtained from selected 
states, districts, and schools reveals important issues and key challenges as states, 
districts, and schools nationwide work to meet the ongoing requirements of NCLB. 

EPRRI researchers used two complementary strategies to collect qualitative data: 
analysis of documents and in-depth interviewing (Marshall & Rossman, 1999). The use 
of two methods of data collection and multiple data sources allowed for triangulation by 
data source. EPRRI staff developed a sound understanding of the history and context of 
educational reform in each of the study sites by ongoing reviews of state and district 
policies available online or obtained directly from the participants. Examples of the types 
of state documents reviewed include Board of Education policies and minutes, strategic 
plans, reports from Superintendents and Commissioners, and state Education Department 
letters to districts. We also reviewed state-consolidated applications for federal funding 
under NCLB and state accountability plans submitted to the U.S. Department of 
Education to comply with NCLB. In addition we collected press releases and letters 
issued by the Secretary of Education, and speeches by the President and Secretary that 
were obtained from the U.S. Department of Education web site. We also reviewed reports 
issued by the U.S. General Accounting Office on NCLB, journal articles and local 
newspaper articles from across the four sites. Finally, we monitored state Web sites for 
information on evolving state efforts to implement NCLB and for state policy documents 
developed to meet the NCLB requirements. 
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The guiding questions for the research came directly from OSEP’S grant priority, 
but were further broken into subheadings and sample issues/indicators. In February 2001, 
the state-level participants and EPRRI staff together reviewed and revised a draft 
template of research questions. This template is available on the EPRRI website, 
www.eprri.org . This process led to the development of 10 individual interview protocols 
focused on the areas of accountability, assessment, monitoring, curriculum and 
instruction, special education, teacher certification, professional development, Title I, 
transition, and parent viewpoints. In October 2001, the same process was used to develop 
interview protocols at the district level. Individual school level protocols were developed 
for the school principal, regular education teachers and special education teachers in late 
2002 . 

Selection of participants at the state and district level was done purposively with 
the support and cooperation of the state and district directors of special education, who 
acted as gatekeepers. At the state and district levels, the special education director 
participated in the identification, initial contact, and interview arrangements with key 
personnel. Knowledgeable personnel from the following departments in each SEA and 
LEA were interviewed: special education, accountability, testing, special education 
monitoring, Title I monitoring, curriculum, teacher certification, and professional 
development. 

EPRRI was specifically concerned with identifying schools within its study sites 
that were high perfonning for students with disabilities compared to other schools in the 
same district. However, the tenn “high performing” is used relatively, as in general the 
performance of students with disabilities lagged behind that of students without 
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disabilities by an average of 30 percentage points. We used one of two ways to identify 
each high-performing school: (a) its school perfonnance index (SPI) or (b) district 
recommendation. 

School Performance Index. The SPI allowed a closer examination of the 
elementary schools in our participating districts and identified schools that were 
relatively high performing for this population. Several factors were considered important 
in the creation of this index. First, we wanted to capture performance across multiple 
years, to minimize the chance of misinterpreting anomalies in perfonnance data. 
Fluctuation in data is especially an issue when examining performance trends in schools 
with low numbers of special education students. In addition to considering performance 
over more than one year, we wanted to include performance across more than one grade 
level and across both reading/language arts and math. By including multiple indicators of 
success, we created an index reflective of “high performance” with respect to multiple 
domains. Most importantly, the index allowed us to identify schools that demonstrated 
some degree of success with all students over a minimum of two successive school years. 

The first step in creating the school index was to sum the performance data for the 
various performance variables described above. In this manner, performance data for 
students with disabilities were weighted equally to performance scores for students in 
general education. We viewed this as appropriate give our specific interest in pinpointing 
and examining schools that demonstrate some level of relative success for students with 
disabilities in addition to some level of overall success. Summed performance scores 
were then standardized to allow for comparison across states. 
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Recognizing the strong correlation between socioeconomic status and 
performance on standardized assessments, we wanted to take the overall socioeconomic 
status of each school’s population of students into consideration when we examined 
index scores. Therefore, a variable reflecting each school’s socioeconomic status 
(School-SES) over the data collection years included in each state’s school perfonnance 
index was also created. The “School-SES” variable was operationalized as the mean 
percent of students receiving free or reduced price meals at each school over the two data 
collection years included in the school performance index, which was then standardized 
and reported as a z-score. Schools with performance index scores above the mean (i.e., z- 
scores at or above zero) that were also above the trend line reflecting the overall 
correlation of school SES to performance were selected for interviews. Schools in 
Maryland, New York, and Texas were selected in this manner. 

Overview of Data Collection Procedures 

During this five-year multi-state study we conducted 35 interviews at the state 
level and 44 at the district level. State level interviews occurred between October 2001 
and January 2002 and district level interviews took place between April and June 2002. 
Once we had identified the schools, the special education director from each LEA 
facilitated the initial contact and we worked closely with the school principal to 
determine when it would be convenient to visit. School visits occurred between 
December 2003 and June 2004. We conducted in-depth interviews with special and 
general educators and the school principals from 20 elementary schools. We requested 
that the individual school principals select teachers for interviews based on the following 
criteria: (a) he or she had worked in the school for two or more years, (b) he or she 
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worked with students in the grades assessed by their state for the purpose of 
accountability, and (c) the general education teacher had students with disabilities in his 
or her classroom during the current academic year. 

The interviews at all levels of the education system were between one and two 
hours in length and adopted a semi-structured approach with the interview protocols 
acting as rough guides. Interviews were also audiotaped with the participants’ consent so 
that they could take the form of a conversation. Finally, the interviews were conducted by 
multiple researchers (Eisenhardt & Bourgeois, 1988), allowing the researchers to overlap 
data analysis and data collection, as recommended by Glaser and Strauss (1967) as well 
as Van Maanen (1988). The use of multiple investigators provided complementary 
insights, added richness to the data analysis, and enhanced confidence in the findings 
(Eisenhardt, 2002). 

Data Analysis 

EPRRI researchers followed the qualitative data analysis procedures 
recommended by Miles and Huberman (1994). After each interview field notes were 
written up by individual researchers, while the taped interviews were transcribed by 
graduate students. At each level of the system individual case study data analysis began 
with the creation of a contact summary sheet to develop an overall picture of the main 
points of each interview. The graduate assistant who first transcribed the interview then 
read each contact summary sheet to identify bias and selectivity. We developed detailed 
descriptive write-ups for each site, based on the field notes, the contact summary sheet, 
and the document reviews. This step in the data analysis is central to the generation of 
insight and helps researchers to cope with the enormous volume of data (Miles & 
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Huberman, 1994; Pettigrew, 1988). 

We then entered the transcripts into a qualitative software program, Ethnograph, 
which allows for the analysis of text-based data into codes and categories of meaning. 
EPRRI researchers adopted a coding approach partway between the a priori and inductive 
approaches discussed by Miles and Huberman (1994). A general coding scheme, based 
on that developed by Bogdon and Biklen (1992), was created to provide a structural, 
conceptual, and coherent order to the emerging codes. This particular coding scheme was 
based on the general domains covered by the research matrices and the interview 
protocols and allowed codes to develop inductively, while at the same time enabling the 
researcher to “think about categories in which codes will have to be developed” (Miles & 
Huberman, 1994, p. 61). 

We created clear operational definitions for each code so that the codes could be 
applied consistently. Code names closest to the concepts being described were applied to 
the chu nk s of data. Initial coding of the data corpus was performed by the first author and 
a team of graduate research assistants, who read and reread each interview line by line 
and coded the sentences or phrases relating to the participants’ perceptions of the effects 
of accountability on students with disabilities and the systems that serve them. EPRRI 
researchers followed the recommendations of Miles and Huberman (1994) for check- 
coding. Two researchers separately coded the first eight pages of the opening interviews 
from each level of the system and reviewed the coded sections together. Intercoder 
reliability was determined using the following fonnula: 

Reliability = number of agreements / total number of agreements + disagreements. 
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Initial intercoder reliability at each level ranged from 79 to 83 percent and rose to 
between 9 1 and 93 percent after the differences were clarified. A further check was 
performed two-thirds of the way through the data analysis. A conceptually clustered 
matrix was developed during the early analysis based on the interview protocols. The 
following decision rules were applied as themes were identified: (a) a theme was coded 
as present for a participant if it was mentioned repeatedly or with strong emphasis during 
the interview; and (b) a theme was coded as present for a study site if it was mentioned 
by two or more participants. 

The second step in the analysis was to put the data back together again in a new 
way to reveal themes and stressors related to the impact of accountability at each level. 
This process is similar to axial coding in grounded theory (Strauss & Corbin, 1990), as it 
involved grouping and comparing the initial codes with each other and merging similar 
concepts together into larger encompassing themes. During this process, all key ideas, 
findings, and interpretations were presented and discussed by EPRRI staff and, at the 
state and district level, with study participants (Miles & Huberman, 1994). Original 
transcripts were used as evidence to support the emerging themes, and at all times the 
researchers searched for examples that would contradict key findings (Yin, 1989). 

II. CALIFORNIA ASSESSMENT AND ACCOUNTABILITY SYSTEMS 

Each of the four states that participated in this research has a unique 
accountability system to determine the academic achievement of their students. 
Information about each state’s accountability system was retrieved from policy 
documents and other sources, primarily via the Internet. Examples of the types of 
documents reviewed include Board of Education policies and minutes; strategic plans; 
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reports from Superintendents and Commissioners; California Department of Education 
(CDE) memos; policies relating to standards-based reform, assessments, and 
accountability; press releases; and other reports. 

Content Standards and Curriculum Frameworks 

Background. Following the publication of A Nation at Risk, California adopted a 
notion of systemic reform in education including curriculum frameworks, assessment, 
and capacity building that changed school governance from local control to state control 
(Timar, 2003). California’s approach to reform was hailed as particularly innovative and 
the state became a recognized leader in the development of state frameworks (Carlos & 
Kirst, 1997). California was one of the first states to develop a comprehensive reform 
strategy that not only broadly outlined what students should know and be able to do in 
core subject areas (Goertz, Flodden, & O’Day, 1996), but also embraced within these 
new frameworks constructivist principles, concepts, and pedagogy. The reformers in 
California emphasized higher order thinking skills, student-centered meaning rather than 
rote memorization, whole language instruction, and interdisciplinary learning. In these 
early stages CDE and the State Superintendent guided reform and established an 
infrastructure of professional development networks run by experienced teacher leaders 
and university professors who worked closely with schools. 

In the early 1990s the direction of education refonn changed radically. Budget 
deficits, political changes in the state, internal squabbling between the State 
Superintendent and the California State Board of Education (CSBE), and abysmal results 
of California students on national tests (Carlos & Kirst, 1997) led to a retrenchment — not 
of the concept of systemic reform, but of the means adopted to get there. Control of 
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education policy was taken away from CDE by judicial action and vested in the CSBE. 
CSBE perceived CDE personnel and the other education professionals who directed the 
earlier reforms to be biased toward unproven content and pedagogy. The language arts 
frameworks in particular received wide condemnation and teachers were blamed for 
taking the whole-language approach to extremes (Carlos & Kirst, 1997). Constructivist 
principles and other progressive approaches that had dominated earlier reforms were 
abandoned in favor of phonics, computation, and other basic skills. Several researchers 
speculate that although the State Superintendent and CDE had a clear and coherent 
vision, the implementation of this vision was problematic as the three prongs of reform — 
curriculum, assessment, and teacher capacity — were not synchronized (Cohen & 

Spillane, 1992; Massell, Kirst, & Hoppe, 1996). 

The direction of state curriculum policy became the territory of the newly elected 
Board of Education officials, while those previously in charge — the State Superintendent, 
CDE, teacher leaders, and curriculum specialists — no longer had a leading role in 
curriculum development. In 1995, the state legislature added another ingredient to the 
mix in the fonn of a separate commission established to develop a new set of grade-by- 
grade content standards and a new assessment program. Over the next ten years authority 
over education policy became widely dispersed and at times contentious, typified by 
ongoing battles between the superintendent and CSBE, and resulted in an absence of 
coherence or consistency (Timar, 2003). 

The battle over curriculum and pedagogy continued among stakeholders, some of 
whom wanted to strike a balance between the new constructivist approaches and more 
traditional approaches, while others wanted to adopt a more back-to-basics agenda. The 
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back-to-basics movement appeared to gain the upper hand with the passage of the 
California Assessment and Academic Achievement Act toward the end of 1995, the 
purpose of which was to develop aligned statewide standards and assessment policies. 
This Act established a 21 -member advisory committee, called the Commission for the 
Establishment of Academic Content and Performance Standards, to oversee the 
development of grade-by-grade standards in core subject areas and to develop an aligned 
assessment program. 

Content standards. The new content standards are organized by grade level, grade 
clusters, or subject matter disciplines and serve as the anchors for the state’s detailed 
curriculum frameworks, instructional materials adoptions in the core curriculum areas, 
and statewide assessment system (CDE, 2006). In November and December 1997 the 
CSBE adopted content standards in English language arts (ELA) and mathematics, and in 
October adopted content standards for history/social science and science. In January 
2001, content standards were adopted for visual and performing arts, and physical 
education standards were adopted January 2005. The CSBE is required to adopt revised 
health standards by March 2008 and foreign language standards no later than June 2009. 

Curriculum frameworks. When the curriculum frameworks were first created 
during the 1990s the intention was that districts and schools would fill in the details to 
create weekly and day-to-day lesson plans (CDE, 1995). However, the 1997 frameworks 
are more explicit about the standards concerning what students must know. Although the 
frameworks are voluntary, they also lay out what students will be tested on and therefore 
what schools will be held accountable for. The frameworks provide broad minimum 
standards and guidelines for educational programs. They directly focus on essential 
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subject matter and provide a foundation for both what teachers are to teach and how they 
are to teach it by describing the scope and sequence of the knowledge and skills that 
students are expected to learn (CDE, 2006). CSBE adopted ELA content standards in 
1997 and these are currently under revision. The mathematics frameworks were revised 
in December 2000 to embody the new content standards, and again in 2005. 

History/social science and foreign language frameworks were revised in 2001, science 
and health in 2002, and visual and performing arts in 2003. CSBE is advised by the 
Curriculum Development and Supplemental Materials Commission (Curriculum 
Commission), which is responsible for overseeing the development of curriculum 
frameworks and recommending them to CSBE. 

Instructional materials evaluation and adoption. The CSBE has the constitutional 
authority to adopt instructional materials for grade 1-8. Instructional materials are broadly 
defined to include textbooks, technology-based materials, other educational materials, 
and tests (CDE, 2006). The instructional materials are designed to be a principal learning 
resource that meets the basic requirements of a full course of study in a given content 
area. The CSBE is advised by the Curriculum Commission and conducts primary 
adoptions every six years for the core curriculum areas of ELA, mathematics, 
history/social science, and science. The remaining curriculum areas are on an eight-year 
adoption cycle. 

Although local school districts also have the authority to conduct their own 
evaluations of instructional materials for high school students and adopt the ones that best 
meet their needs, they must be in compliance with the California Education Code and 
CSBE guidelines. The Instructional Materials Funding Realignment Program (IMFRP) 
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became law on January 1, 2003 and requires that school districts ensure that each student 
in kindergarten and grades 1-8 is provided with a standards-aligned textbook or other 
basic instructional materials as adopted by CSBE before they can purchase other CSBE 
adopted materials (CDE, 2006). 

Content standards, curriculum frameworks, and studen ts with disabilities. 
Although statements in the content standards and curriculum frameworks documents 
assert that they apply to all children, there is little evidence that the needs of students with 
disabilities were explicitly considered when the content standards and curriculum 
frameworks were originally developed. Beginning in 1999, when CSBE adopted new 
textbooks and instructional materials, publishers were required to design textbooks and 
instructional materials that fostered universal access to the curriculum. For example, in 
the ELA adoption criteria for K-8 all publishers were asked to provide additional 
instructional materials for special populations of students including those significantly 
below grade level and English language learners (CDE, 1999). 

In October 2001, as a response to IDEA 97, an Alternate Assessment Advisory 
Workgroup (AAAW) reviewed the California content standards in ELA, mathematics, 
science, and history/social science and identified those standards most appropriate for 
students with significant disabilities. From this subset of content standards a 
representative group of California teachers developed a list of functional perfonnance 
indicators for each standard. Although the state expects districts and schools to provide 
access to the complete range of California content standards, they are held accountable 
for student progress only on this subset of standards. 
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Assessment 



Background. The history of the development of assessments in California mirrors 
the often tumultuous nature of the development of content standards and curriculum. 
California’s first significant assessment program was developed in 1972 (Carlos & Kirst, 
1997). The California Assessment program (CAP) was a multiple-choice instrument that 
was administered to a sample of students in each school. The primary purpose was to 
measure local system and school effectiveness. CAP did not produce individual scores 
and therefore did not provide teachers with student-by-student scores to guide 
instructional practice. Funding for CAP was eliminated, apparently due to a political 
squabble between the Governor and the State Superintendent (Carlos & Kirst, 1997). 

In 1991 CDE developed the California Learning Assessment System (CLAS), 
which included both traditional and nontraditional testing methods. CDE was the prime 
force behind CLAS and hoped the tests would be effective both as accountability tools 
and to guide instruction. Although CDE had support from some segments of the 
educational community CLAS was not without some powerful critics, including a vocal 
group of parents who were especially critical of the literacy assessments. After the first 
administration of CLAS critics of the assessment expressed doubts about the technical 
adequacy of the assessment. Amid this criticism the state developed an interim voluntary 
testing program, the Pupil Testing Incentive Program (PTIP). PTIP provided districts 
with five dollars per student as an incentive to administer tests from a CSBE approved 
list. 
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California State Assessments 



The Standardized Testing and Reporting Program (STAR). The Standardized 
Testing and Reporting Program (STAR), was authorized in October 1997 and replaced 
the PTIP. Under STAR, California state law allows a parent or guardian to submit to the 
school a written request to excuse his or her child from any or all parts of any test. A 
school district and its employees may discuss STAR with parents and may inform parents 
of the availability of exemptions, but should not solicit or encourage any written 
exemption request on behalf of any child or group of children. 

In November 1997, the Stanford Achievement Test Series, Ninth Edition, Fonn T 
(SAT9) was chosen as the STAR test for a period of five years. The SAT9 is a multiple- 
choice test that allowed comparisons of scores from a national sample of students. School 
districts in California were required to test all students in grades 2 through 1 1 . Students in 
grades 2 through 8 were tested in reading, mathematics, written expression, and spelling. 
Students in grades 9 through 1 1 were tested in reading, writing, mathematics, science, 
and history/social science. 

However, the state’s adoption of SAT9 did not meet federal requirements under 
the Improve America’s Schools Act (IASA) because it was not aligned with the state’s 
content standards. In response, in 1998 CSBE authorized the development of standards- 
based tests in ELA and mathematics, grades 2 -1 1, to augment the SAT9 and to be 
administered the following year. In 2000, CSBE authorized the development of 
additional standards-based tests in writing at grades 4 and 7 and history/social science 
and science for students in grades 9 through 1 1 . These assessments became known as the 
California Standards Tests (CSTs) and largely eclipsed the norm-referenced assessment 
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in the state’s accountability system. In 2001 the STAR program was reauthorized and in 
2002 CSBE designated Educational Testing Service (ETS) as the contractor for the 
California Achievement Tests, Sixth Edition Survey (CAT/6). The CAT/6 replaced both 
the SAT9 and the CSTs. ETS’s contract covered 2002-03, 2003-04, and 2004-05 and in 
2004 was extended for an additional year to cover 2005-06. CSBE is currently 
considering proposals from four test contractors to administer the STAR program from 
SY 2006-07 to SY 2008-09, continue to develop the CSTs and the California Alternate 
Performance Assessment (CAPA), and to begin development on a California Modified 
Assessment (CMA) and standards-based tests in Spanish. 

Assessment and students with IEPs. CAPA was developed to meet IDEA 97’s 
requirement that all students with disabilities be included in state and district testing. This 
assessment was designed by Educational Testing Services and was administered for the 
first time in Spring 2003. CAPA is part of the STAR Program and is available to students 
with more significant disabilities in grades 2-11 who are unable to take a large-scale 
assessment, even with accommodations. CAPA is an on-demand performance event 
assessment aligned to a subset of the California content standards, specifically the ELA 
and mathematics standards. 

The CAPA has five levels: Level I (grades 2-11) Level II (grades 2-3); Level III 
(grades 4-5); Level IV (grades 6-8) and Level V (grades 9-11). Most students eligible for 
the CAPA will take the level corresponding to their grade. However, some students with 
complex, profound disabilities may be eligible only for Level I. This level represents an 
opportunity for students with the most significant disabilities to demonstrate their skills 
and independence. For those students with severe disabilities who are not assigned to a 
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grade level, subtracting 5 from the student's chronological age determines the student's 
grade for testing purposes. For example for accountability purposes, a 12 year old would 
be enrolled in grade 7 in a graded program. As with the STAR assessments, a student 
may be exempted from CAPA by parental waiver. 

California also allowed some students with disabilities who were not eligible for 
the CAPA, but who also could not participate in STAR assessments on grade level, to 
take an out-of-level CST. However, out-of-level testing was not permissible for students 
in grades 2, 3, or 4 and any student taking an out-of-level test was required to participate 
in all the assessments given at that grade level (CDE, 2003). In addition, students could 
be tested no more than two grade levels below their enrollment grade. Interestingly, 
California allowed students with IEPs or Section 504 plans in grades 5-11 to take out-of- 
level assessments. However, beginning with the 2006 spring administration out-of-level 
testing was no longer allowed and student IEPs and Section 504 plans were modified to 
reflect this change (CDE, 2006). 

Further changes to the STAR program, related to the assessment of students with 
IEPs, are planned in the near future. For example, because CAPA was initially developed 
as a response to the IDEA 97 requirements it does not meet NCLB requirements 
concerning linkage to state academic content standards by grade level, and is in need of 
revision (Jack O’Connell, personal communication, 2005). In addition, California plans 
to develop an alternate assessment based on modified achievement standards for students 
with moderate cognitive disabilities, who were previously assessed using the CST with 
modifications. The California Modified Assessment (CMA) is scheduled for field testing 
in 2007 and will be operational in Spring 2008. 
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California High School Exit Examination (CAHSEE). In 1999 the state legislature 
authorized the development of the California High School Exit Examination (CAHSEE) 
in ELA and mathematics. Passing the CAHSEE is one of the requirements for graduation 
with a high school diploma. The purpose of the CAHSEE was to set higher standards for 
high school graduation and to ensure that students who graduate from high school can 
demonstrate grade-level competency. However, while the ELA examination is aligned 
with content standards for ninth and tenth grade, the mathematics examination is aligned 
with the content standards for sixth and seventh grade. Students who fail the CAHSEE 
must retake the examination until both the ELA and mathematics parts are passed; 
however, students may retake only those parts not previously passed. The first 
opportunity students have to take the CAHSEE is in the second half of grade 10. Students 
who do not pass the CAHSEE during this administration have up to five additional 
opportunities to pass the CAHSEE. 

Originally, the CAHSEE was to become a requirement for a state high school 
diploma for the graduating class of 2004. Following several mandated independent 
evaluations of the CAHSEE (Wise, Demeyer, Harris, Sipes, Hoffman, & Ford, 2000), 
and in response to concerns that students would not have sufficient opportunity to learn 
the content, the SBE took action in July 2003 to move this diploma requirement to the 
Class of 2006. However, in February 2006 a one-year exemption of the requirement to 
pass the CAHSEE was provided by the legislature for certain students with disabilities (J. 
O’Connell, personal communication, February 3, 2006). Eligible students must have an 
IEP or a Section 504 plan, be scheduled to receive a high school diploma, meet other 
requirements for a high school diploma, have attempted to pass the CAHSEE at least 
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twice, and have received remedial or supplemental instruction. 

Implementation of the CAHSEE requirement for a high school diploma has 
proved particularly problematic for students with disabilities whose IEPs or Section 504 
plans included assessment accommodations and modifications. The state of California 
found itself embroiled in a CAHSEE-related lawsuit in May 2001 ( Juleus Chapman et al. 
v. California Department of Education). The state initially indicated the students with 
disabilities would have to request a waiver to use accommodations or modifications 
during the exit exam. This was quickly stopped by the judge who heard the case. An 
immediate solution was imposed by the judge, who indicated that all students with 
disabilities who needed accommodations to participate in the exit exam could receive 
whatever accommodations they needed. The state established a state-level appeals 
process for students who received the equivalent of a passing score while using 
modifications on the CAHSEE, to enable them to graduate with a high school diploma. 
This appeals process was subsequently moved to the local level in 2003 (CDE, 2004). 

The state also established (in response to legislative and state board actions, SB 
964) an advisory panel to consider options and provide input to identify 
recommendations for alternatives to the high school exit exam for students with 
disabilities. These alternatives are expected to provide different ways for students to 
demonstrate that they have met the graduation standards when they need 
accommodations that would produce invalid scores for the exit exam. 

Calif orn ia A ccoun tabii i ty System 

Immediately upon becoming governor in 1999, Gray Davis called a special 
legislative session in January 1999 to enact an entirely new school accountability scheme. 
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This new system was developed in a matter of weeks (Timar, 2003) and was 
implemented September, 1999. Accountability in California’s educational system is 
defined by the Public Schools Accountability Act (PSAA) of 1999, which authorized the 
creation of a new educational accountability system for California public schools. The 
Public School Perfonnance Accountability Program consisted of the following three 
components: (a) the state Academic Performance Index (API); (b) the Immediate 
Intervention/Underperforming Schools Program and (c) the Governor's Immediate 
Achieving/Improving Schools Program. California reports its API as: (1) the base 
component and (2) the growth component. The base report is published each January or 
February to inform schools on the amount of progress they need to make on that year’s 
Spring assessments to meet their individual API target. In the fall, the growth reports are 
published to show whether schools have reached their growth targets based on 
performance on the previous spring’s assessments. 

The State Board of Education has established five performance levels for the state 
assessments: advanced, proficient, basic, below basic, and far below basic. Students 
performing at the proficient level, the state’s desired performance level for all students, 
score at or above the 50 th percentile on the CAT/6. The California Standards Test (CST) 
requires the individual to reach a score of 350 or higher and the CAPA requires a score of 
35 or higher to be at the proficient level. These definitions of proficiency are used to 
calculate Adequate Yearly Progress (AYP) under NCLB. 

Academic Performance Index (API). The cornerstone of California’s Public 
Schools Accountability Act of 1999 is the API. This figure is used to measure the annual 
growth and academic performance of schools by assigning a numeric value, ranging from 
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200 to 1000, to their academic improvements; the target score for all schools is 800. The 
weight of each content area test in the school’s API varies by elementary/middle school 
and high school (CDE, 2004). 

To calculate a school’s API score, each student’s national percentile rank on the 
norm-referenced test, standards-based performance level on the CST, and at the high 
school level the performance level on the CAHSEE, are weighted and combined to 
produce a summary result for each area. Those summary results are then also weighted 
and combined to yield a number between 200 and 1000, which is the school’s API. Prior 
to July 2003, the minimum annual growth target for each school with an API lower than 
800 was 5% or a minimum of one point. Schools with an API above 800 must maintain 
that API. 

California also requires comparable improvement at the school level for 
numerically significant subgroups. In 1999, to be considered a numerically significant 
subgroup, a group needed to have at least 30 students with valid test scores, who made up 
at least 15 percent of the school’s valid test scores, or at least 100 students with valid test 
scores. In 2004 this definition was changed to instead include groups that had at least 50 
valid test scores, with these scores making up at least 15% of a school’s valid test scores. 
Each numerically significant subgroup at the school must now achieve at least 80% of the 
school- wide annual growth target. In 1999, PSAA named the following subgroups: 
African American or Black, American Indian or Alaska Native, Asian, Filipino, Hispanic 
or Latino, White, and Socio-economically disadvantaged. A socio-economically 
disadvantaged student was defined as a student who received free or reduced meals or a 
student whose parents have not received a high school diploma. In 2005 CSBE added 
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English language learners and students with disabilities to the subgroups used in API 
calculations. 

According to the Public Schools Accountability Act of 1999, if a school meets 
their API growth criterion, the school is eligible for the Governor’s Performance Award 
Program, which includes monetary rewards, special commendations and school honor 
roll; or the Certified Staff Performance Incentive Award. Schools ranked in the lower half 
of the state scores and not meeting their growth targets will be identified for the 
Immediate Intervention/Underperforming Schools Program. 

Immediate Intervention/Underperforming Schools Program (II/USP). If a school 
fails to meet its API growth target, the school is recommended for the II/USP and it could 
be selected to receive improvement funding. If the school fails to meet their API growth 
target in the following year, local interventions will be put into place such as removing 
the barriers toward improved student achievement or requesting a High Priority Schools 
Grant from the state amounting to $200 per child and not less that $50,000 per school. If 
local interventions do not help the school to meet its target in the third consecutive 
reporting cycle, the school is deemed low-performing and the “Superintendent of Public 
Instruction shah assume ah the legal rights, duties, and powers of the governing board 
with respect to that school” (Summary of SB 1552, 52055.5c, 1999). The State Board of 
Education then has the option of at least one of the following: 

• Revising the attendance policy for the students, allowing them to attend any 
public school with space; 

• Allowing parents to form a charter school; 

• Assigning the management of the school to a university or other high education 



25 




professional; 

• Reassigning employees of the school; 

• Renegotiating the teachers’ contract at the end of the contracted year; 

• Reorganizing the school; or 

• Closing the school. 

The API aligns with the AYP requirement of NCLB. Progress of one point or 
more on the API will translate to the same amount of gain in the AYP calculator. At the 
elementary level, scores from the CAPA have been aligned with the performance levels 
from the CST; the CAPA scores can be integrated with those of the students assessed 
with the CST to produce one measure of proficient for calculation of a school’s AYP. At 
the high school level, the CAHSEE is scored as “pass/no pass.” To meet NCLB’s 95% 
participation requirements for AYP, participation rates in all tests have been determined 
by requiring that all students complete the demographical portion of the Student Answer 
Document whether or not they are participating in the standard assessments. However, 
the U.S. Department of Education informed the state that they were no longer allowed to 
consider students who use modifications as having participated in the federal 
accountability system to meet AYP. Therefore, beginning in SY 2005-06 students who 
use modifications on the CSTs will not count toward NCLB participation rate (J. 
O’Connell, personal communication, October 2005). 

All schools in California have been advised that they must improve by annual 
increments of 10.8 in their ELA scores before 2014 and that they must improve by 
increments of 10.5 in their mathematics scores by the year 2014. However, there have 
been no changes to the consequences that a school will face for failing to meet its AYP 
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goal since the adoption of NCLB. According to the most recent information available, 
California is continuing to discuss how best to align its present state interventions for low 
performing schools with the specific requirements of NCLB. This will require extensive 
legislative and regulatory changes. 

Before NCLB, California required that schools be accountable for a child’s 
performance once the child had been in that school district for a year. This has been 
changed so a student must be in a school for a year’s time before the district is 
accountable for the student’s performance. At this time, California does not have a way to 
longitudinally track high school students from entry to graduation, as required by NCLB, 
but like many states is in the process of developing one. 

III. CALIFORNIA STATE AND LOCAL DISTRICT DEMOGRAPHICS 

In the 2000-01 school year California enrolled just over six million students and 
identified almost 47% as economically disadvantaged (See Table 1). Just over 10% of 
California students received special education services and 25% received English learner 
services. In SY 2000-01 about 43% of California’s students were Hispanic, almost 36% 
were White, just over 8% were African American, and the remaining 12.5% were Pacific 
Islander, Asian, or American Indian. In addition, the percentage of children receiving 
Free and Reduced Meals (FARMS) increased to almost 50% (46.8%). The average 
student-to-teacher ratio was 20.7 students to 1 teacher and the average operating 
expenditure per pupil was $6360 (see Table 2). 
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Table 1 



State and district select demographics 2000-01 



State/ District 


Enrollment 


% FARMS 


% Students 
Receiving Special 
Education 
Services (k-12) 


% Students 
Receiving 
English 
Language 
Services 


% African 
American 


Ethnic Group 
% Hispanic % White 


% Other 


California 


6,050,895 


46.8 


10.4 


25.0 


8.4 


43.2 


35.9 


12.5 


Long Beach 


93,694 


68.3 


1A 


33.8 


19.7 


45.4 


17.8 


17.1 


Unified 


















New Haven 


13,775 


26.7 


8.9 


25.0 


9.8 


29.8 


19.8 


40.6 


Unified 



















FARMS= Free and Reduced Meals 
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Table 2 



Selected state and school district staff and financial statistics, 2000-01 school year 



State/School District 


Average Number of Students/ Teacher 


Total Expenditure per pupil for general education 


California 


20.7 


$6,360 


Long Beach Unified 


21.0 


6,516 


New Haven Unified 


20.0 


6,583 
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Long Beach Unified School District (LBUSD) is a large urban district just outside 
Los Angeles. It educates more than 93,000 students in 95 public schools in the cities of 
Long Beach, Lakewood, Signal Hill, and Avalon on Catalina Island. It is the third largest 
school district in California and serves the most diverse large city in the United States; 46 
different languages are spoken by local students. The district employs more than 8,000 
people, making it the largest employer in Long Beach. The school district has won 
widespread recognition for establishing high standards of dress, behavior and 
achievement, and was named the 2003-04 national winner of the $500,000 Broad Prize 
for Urban Education, which recognizes America's best urban school system for 
increasing student achievement. Long before the strategy became common among U.S. 
schools, LBUSD developed clear expectations for what children should know and be able 
to do at each grade level. The district in the early 1990s looked to teachers, business 
leaders, university experts and parents to develop new and rigorous academic standards. 

About 68% of Long Beach students were economically disadvantaged in school 
year 2000-01, a higher percentage than that of the state overall. About 7% of Long Beach 
students received special education services and 34% received English language services. 
Over 45% percent of the students were Hispanic, nearly 20% African American, 18% 
were White, and the remaining 17% including Asian, Pacific Islander, Filipino, or other. 
The number of students per teacher (21:1) and the operating expenditures per pupil 
($6516) were slightly than the state average. 

New Haven Unified School District (NHUSD) is located in the San Francisco Bay 
Area and serves Union City and south Hayward. The district was created in 1965 and 
now includes over 13,700 students attending the district's 12 schools. There are eight 
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elementary schools with grades K-5; three middle schools with grades 6-8; and one high 
school. However the district is facing declining enrollment as families with children leave 
the area in search of affordable housing. Over 26% of the students in NHUSD were 
economically disadvantaged (i.e., received Free and Reduced Meals - FARMS), 
considerably lower than that of the state’s as a whole. Almost 9% of the students in 
NHUSD received special education services and around 25% received English language 
services. NHUSD had a diverse student body of almost 30% Hispanic students, almost 
20% White students, 10% African American, 21% Asian, and 17% Filipino. The latter 
two groups made up the “Other” category in Table 1. The number of students per teacher 
was 20 to 1, which was close to the state average, with an operating expenditure per pupil 
of $6583, higher than the average for the state as a whole. 

IV. CALIFORNIA: EMBEDDED STATE AND DISTRICT FINDINGS 

As discussed in Section II, California adopted the principles of standards-based 
reform at the state level during the early 1990s, but these reform efforts became mired in 
a series of political controversies and turf fights that greatly tarnished initial and 
continuing reform efforts. The central feature of the controversy was not a debate about 
standards-based reform per se, but rather the control and implementation of the reform 
policy. It was evident during our interviews with CDE staff that this struggle continues 
and provides a backdrop against which further changes at the state and federal levels 
must be viewed. 

State-level interviews took place in October and December of 2001, and district 
interviews took place in June of 2002. EPRRI staff interviewed ten individuals at the 
state level and at LBUSD and at NHUSED. Findings are arranged into three broad topical 
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areas: 1) fitting students with IEPs into the turbulent policy environment, 2) building 
system capacity, and 3) shifts in the locus of control over education. In each broad area, 
the data are presented at the state level and then the district level, and the significant 
conceptual themes that emerged are illustrated by a small number of exemplar quotes. 
Each quote chosen is representative of many such instances, because it is an extreme 
instance or the sole instance of something, or because it represents two contrasting 
properties of the conceptual theme under discussion. 

Fitting Students with IEPs into a Turbulent Policy Environment 

When the state level and district level interviews took place in October and 
December 2001 and June 2002 respectively, our informants were concerned with making 
sense of the ongoing changes in policy and the recently created STAR system. Although 
we were primarily concerned with the impact of accountability reform on students with 
IEPs, it was clear that our informants were only just beginning to consider how these 
students could be included in the state’s evolving system. Indeed, there was considerable 
evidence that this population was rarely considered by general education policymakers at 
all. One participant commented: “Sometimes they’ll let the at-risk folks come to the 
table, sometimes they’ll let the English language learner folks come to the table, and very 
infrequently did they ever let Special Ed come to the table and this has been a historical 
problem in California and I suspect in most states.” In the next section we will explore 
the following issues: participation in assessment, exclusion from the API, and 
problematic implementation of the high school exit examination for students with IEPs. 
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When we visited the state in 2001 the STAR system was still under construction 

and policy decisions were being made rapidly — possibly too rapidly according to several 

of our participants. One participant stated: 

Now, we’re not exactly sure how the Standards stuff is going to work and you 
know you get pushed into doing something faster than you really want to do it and 
we just don’t have a lot of experience. We did as much modeling as we could on 
what we had, but.... 

Another infonnant added: “Frankly the assessment system has changed constantly so I 

really can’t say if it’s out of sync because it’s just been evolving over so much time.” 

Another participant explained it was difficult to maintain public confidence in the API 

system with such rapid changes taking place: 

The thing is, the issues for STAR in California are really changing but it is crucial 
that we maintain some kind of continuity in the mind of the public and the schools 
so they still understand what it is and how it works and they have confidence and 
that’s not real easy. 

As the following quotation illustrates, some CDE staff felt that they, like many states at 
the time, were in a crisis mode and could only react to events as opposed to acting 
proactively: 

All states, we’re all experiencing the same level of crisis and that’s really what we 
feel like we’re doing. We’re a moving target in a crisis mode and you’re putting 
out fires. It’s really hard at the same time to look at some proactive front-loading 
of things to stop the crisis after-effects. 

Issues of Participation and Exclusion 

Evidence from CDE indicates that they wanted as many students with disabilities 
as possible to participate in the STAR system, and developed several options to enable 
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them to participate. As the following quotation indicates, options included taking the 

regular assessment, taking the regular assessment with accommodations, taking the 

regular assessment with non-standard accommodations, taking an out-of-level assessment 

or taking the state’s alternate assessment, which was still under development: 

The special education students have a variety of options when they take their 
STAR assessment. We want as many kids as possible to take the grade-level test 
with as few non-standard accommodations as possible, some take it with non- 
standard and they can also take an out-of-level assessment. Then there are kids 
that just don’t take any test at all in terms of STAR as the alternative assessment 
is not ready yet. 

However, for students with IEPs, while these options enabled them to participate 

in the assessment process, some of the options had implications for how scores could be 

used for accountability purposes, an issue not lost on the school districts. 

Taking the SAT9 with non-standard accommodations. The central issue was that 

the rules concerning a useable score on the norm-referenced assessment did not allow 

most of the accommodations that students with IEPs typically need, such as extra time, 

using a calculator, and having the reading portion read to them. The result was that if a 

student was given a non-standard administration, his or her raw score could not be 

transformed into a scaled score and could not be included in the API. As the following 

quotation illustrates, most students with IEPs participated in the testing process, only to 

have their scores removed from the accountability index: 

Special education kids are expected to take the SAT9 and most do. The question is 
whether you take it as a “standard administration,” that is within all the rules that 
are laid out by Harcourt to define what a valid administration is. But if they have a 
non-standard administration right now they are not a part of the accountability 
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system. 



It was difficult to gauge exactly how many scores were removed due to non- 
standard accommodations, but an analysis of Harcourt scoring rules indicated that they 
were very strict in their interpretation, which results in quite a large number of 
exclusions. In addition, the assessment division sent a memo addressing this stating it 
would not issue an API to schools where too many students received non-standard 
accommodations, “The accountability division has put out that they will not give an API 
if more than 50 percent of special education students have non-standard accommodations 
unless the school can justify why that’s the situation.” This action suggests that the 
practice was widespread enough to cause concern at the state level. 

One of participants expressed frustration with the stringent rules, pointing out 
some of the disallowed accommodations did not in his view change the construct that was 
being tested: 

When you buy a product it comes with this disclaimer: it doesn’t apply to this, 
and it doesn’t apply to that and we don’t know about this or that. I mean they also 
say: “Oh you can’t change the order of the items because it might change the 
norms.” Oh come on guys; it’s the same test. 

Comments from several informants at the state and district level indicated that the 
removal of such scores worked to the advantage of schools. According to this informant 
from one of the participating districts, principals had expressed concerns over the state’s 
requirement that to comply with IDEA 97 all students in special education must 
participate in the state assessment. Principals were particularly concerned about the 
students who were between the regular assessment and the future alternate assessment 
and feared that their scores would be lower. However, because non-standard scores were 
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removed this was unlikely to be an issue: 

Principals have said, ‘Well, does this mean my scores are going to be lower 
because these students are taking this test and if so I don’t want that to happen.’ 
They didn’t need to worry. In this system any student who takes a non-standard 
accommodation — frequently it’s more time — that’s then scored, but it’s not 
included in the results and as it links to the accountability index it’s not used in the 
calculation. 

However, comments from our state-level informants revealed that with the 

introduction of the California Standards Tests and the downgrading of the normative 

assessment, more scores from students with disabilities would be included in the API: 

We have them in all four areas and the plan is to bring all of those phased in over 
the two to three year period. So English Language Arts will be the first, Math will 
come in 2002 and Science and Social Sciences in 2003. My understanding is that 
the standards-based tests literally have no circumstances in which a score is 
invalid. There are no accommodations that render a score not countable. All scores 
will go in. 

Although our informants believed that the introduction of the CSTs was very 
positive, several state level informants identified two specific concerns that they had. The 
first issue was that schools may try to exempt students with disabilities from state 
assessments and the second was that they would assign students to out-of-level CSTs. On 
the first issue informants revealed that some schools did not understand the state’s 
participation policy and believed that IEP teams could exempt students from participation 
in state assessments. Our infonnants explained that schools needed to better understand 
the role of the IEP team in assessment decisions, that is, that the IEP team determined 
how a student with disabilities will be assessed and not whether they will be assessed. As 
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the following quotation illustrates, an IEP team may not exempt a student with a 
disability from state testing. Instead, the role of the IEP team is to determine how the 
student would participate. The only exemptions are those requested by parents — an 
option available to all parents. 

We’ve tried to be clear in directions that an IEP team may not exempt a student 
from any of the tests. That the role of the IEP team is to identify how the student 
will participate and that that participation may be the test with no 
accommodations, the standard accommodations, the non-standard 
accommodations or with the alternate. But that we do expect that the student will 
be assessed. The only legitimate exemptions are the parent requests and those 
parent requests are open for any parent — disabled student or non- disabled 
student. 

Another participant explained that the state had already found evidence that IEP 

teams may face pressure from schools to exempt students in special education, and the 

state intended to request that the division of special education investigate certain schools. 

As one informant at the state level pointed out some schools, especially high schools 

exempted almost all their students in special education: 

Last winter, we did a comparison of the percentage of kids that were reported 
special ed, in the Fall, and the number of kids that actually took the STAR Test. 
There are some schools that exempt a large percentage of their kids from being 
tested, and that issue has been taken up by the special education division. We’re 
concerned about that because there are some high schools, for instance, that — all 
their kids were exempted and that didn’t seem very likely. 

When we visited the school districts in June of 2002 we found evidence from the 
two school districts that revealed a mixed pattern of compliance with the state policy 
regarding IEP exemptions. As the following quotation shows, one district followed the 
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state policy closely and did not allow IEP waivers: 

Basically, there is no exclusion for IEP. You cannot exempt in this district, but I 
think there are all forms of interpretation to that. In this district, you cannot 
exempt a kid. The other thing is that if you want to excuse a kid it is the same 
process as if a general ed parent wants to excuse a kid. It is not an IEP team 
decision. 

A participant from the same district described the origin of the confusion over 

exemptions and students with disabilities. At the state level, the policy was that students 

with disabilities must participate in state assessments and if a student with disabilities 

was exempted it meant they were exempt from standardized testing — not from being 

tested at all — and must take an alternate assessment: 

There were a lot of questions about what does this all mean especially for students 
with disabilities. Generally, what it means is that at the state level, all students are 
to be tested. There were some misunderstandings and even some confusion, 
because an IEP might say, ‘Student exempted from testing’ and we had to 
explain, ‘You are not really exempted. You are a part of what this is. If you are 
exempted means, you are exempted from a standardized norm referenced test, and 
this is where an alternative assessment would come into place’. 

Another infonnant from this district explained that they had really tried to work with 

teachers to decrease the number of non-standard accommodations. Interestingly, this 

participant commented that she knew the state was concerned that district may try to take 

advantage of the impact of non-standard accommodations with the purpose of removing 

low scores. However, such practices did not occur in her district: 

For the state accountability system we just continue this discussion around what is 
the appropriate accommodation for students, so when we can start moving kids 
into the areas where they receive standard accommodations that can be included 
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in the accountability system. I know certainly there is concern and even, I would 
say, criticism from the state level that schools are hiding their low performing 
kids by identifying them as special education and providing non-standard 
accommodations, but I certainly haven’t seen that here. 

On the other hand, a participant from EPRRI’s other district indicated that they 
implemented the policy a little differently. According to this district- level participant, 
IEP teams were advised to exempt certain students with disabilities from the state 
assessment. Students who were placed in special schools, generally those with severe 
impairments, were routinely exempt, while others in separate classes within the regular 
school participated with non-standard accommodations, “What we’ve said is moderate to 
severe special day type kids we recommend in general don’t take it. Then our regular 
special day kids take it with non-standard accommodations and resource kids take it with 
standard accommodations.” This informant reflected that the district’s interpretation of 
the state policy was “humane,” as it was unfair to have a student take an assessment and 
then fail it: “We’ve tried to be somewhat humane about testing because it doesn’t do 
much good for a kid to take a test on grade level and flu nk it.” Another participant from 
this district expressed similar views, calling the state assessments silly: “It’s a very 
complex situation with all the standardized testing and special education students. I mean 
they just have to sit there and take those silly things.” 

Unfortunately, this is a very traditional approach that reinforces the belief that 
students in special education cannot learn to the same standards as other students and is 
in direct conflict with the principles of standards-based reform because of its negative 
consequences. Indeed, laws such as NCLB were designed to challenge the belief that this 
population cannot achieve in a standards-based system. Interestingly, one of our 
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informants, although much opposed to group achievement tests for this population, 
reflected that the test scores of students with disabilities at the state level were 
surprisingly good: 

I personally feel that it doesn’t make a lot of sense for a lot of kids who are 
special education. It’s just a group achievement test and they don’t really need 
that. But I was interested to look at the state scores which I thought were not too 
shabby, for kids who were special ed kids. 

Out-of-level testing. Students with disabilities were also allowed to take the SAT9 
out of level. However, as the following quotation illustrates, this option also had an 
impact on the validity of the score and its inclusion or exclusion from the API. Thus if the 
test was one grade level below, the score was included in the API. If, however, the 
assessment was two or more levels below the student’s enrolled grade, the score was not 
included in the API: 

We also allow kids to take tests out of level. If you take one grade level below 
then those results are reported as part of the school report on the assessment 
system and included in the accountability index. If it is more than one level it is 
reported to the student, but not reported as part of the school score or reported in 
API. 

Other infonnants pointed out that out-of-level testing was problematic as it 
provided schools with an incentive to put pressure on IEP teams to assign students with 
disabilities to an out-of-level test when they should have a grade-level test in order to 
make the school look better. As the following quotation illustrates, CDE was still trying 
to find a solution for the CST : 

I think the dilemma that we’re all in right now is the students who legitimately 
need an out-of-level test to determine what they can do and then there are those 
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students where — the term I use is game playing is going on. The student really is 
capable of taking a grade-level test and they’re capable of showing us what they 
know on a grade-level test. The perception in schools is: ‘If I test them out of 
level, they’ll get a higher score and it’s going to help my school on the 
accountability measure. That’s the piece that we’re kind of caught on — how do 
we fix that? 

Another infonnant, while conceding that this situation may occur, expressed the 

belief that such behaviors were the exceptions and not the rule. As the following 

quotation shows, this informant relied on the IEP process, parent knowledge, and 

vigilance to identify when the needs of the school were put before the needs of the child. 

The closing line is said somewhat wistfully: 

I want to think that when you get to the IEP situation and you’re working with 
parents, it seems to me it would be very difficult and would be somewhat 
transparent to a parent, if you’re just trying to use my child for the benefit of the 
school that may not have anything to do with what’s right for his or her 
instructional program learning. I want to believe that what we’re talking about in 
terms of trying to manipulate or using this student would be the exception and not 
the rule. 

Implementing the CAHSEE for Students with IEPs 

When we visited the state in 2001, the CAHSEE was the dominant issue for all 
informants as the actions of the CSBE had precipitated a high profile legal case brought 
under several disability statutes including IDEA 97. We were unable to interview 
members of the CSBE and therefore have only the opinions of CDE staff to help 
understand the Board’s decision-making process. According to one informant, the CSBE 
did not understand the implications of their decision to forbid certain accommodations: 
The Board has limited staff to be able to really deal with these issues — to think 
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them through. I mean this stuff is so complicated and a single word can have a lot 
of different connotations and meanings that they don’t necessarily really get. 

However, there is considerably more evidence to support the view that at least 
some members of the board knew that their decision on accommodations could 
potentially create a legal headache, but were prepared to follow their philosophical 
viewpoints. Comments from our infonnants reveal considerable tension between CDE 
and CSBE. It was clear that CDE disagreed with the actions of the board, but were 
overruled. According to several informants at CDE, the goal was to enable as many 
students with disabilities as possible to participate in the CAHSEE. To support this goal 
the CDE recommended that a range of accommodations be allowed on the CAHSEE, 
including the use of a calculator and allowing read-aloud for the reading portion of the 
examination: 

We wrote regulations that said if you are blind and you don’t read Braille then you 
can have the test read to you, or if you are dyslexic. If you have computational 
problems you can use a calculator. We thought, get as many kids taking the test as 
we possibly can. Our state board said no, we can’t approve a test where kids are 
allowed not to read. The fundamental act of reading is one of the constructs and so 
we are not going to approve your regulations, we are going to say you can’t have 
it read to you, you cannot use the calculator. 

According to several participants, CSBE did not share the same understanding of 
“disability” as our informants. The state board, or more specifically certain members of 
the state board, believed that many students who had problems reading did not have a 
disability, but rather had not received the necessary instruction: 
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The state board is looking at the message that they are trying to send to the 
system. They believe that there are lots of dyslexic kids that can be taught to read 
and that if you put a lot of pressure on schools the system will change. 

Another infonnant commented that one of the CSBE members “will parrot 
children who are in special ed because they haven’t been taught to read.” An informant 
from CDE expressed considerable frustration at the actions of CSBE, and revealed that he 
had tried to explain the purpose of CAHSEE to members of CSBE and had pointed out 
the doubtful legality of their actions, but that CSBE moved ahead despite knowing that 
this was a civil rights issue: 

I kept trying to say that the exit test it is not designed to change the system. It is 
designed to follow what is happening out there and that there are all sorts of legal 
requirements that kids have the opportunity to leam the material on the test. So 
anyway, they did adopt their regulations that specifically disallowed the use of a 
calculator or having the test read to you or audio presentation. They knew that 
they were creating themselves a civil rights issue. I really felt like they made a 
mistake. We’re being dragged into a lawsuit and it certainly doesn’t help. 

Another infonnant confirmed the information presented above and again 
expressed frustration at CSBE’s actions, and one board member in particular who helped 
create the dilemma by commenting: “Just open the purse and give them the money, this is 
silly stuff.” This informant also explained that in the class action suit the students 
involved had received the specific accommodations throughout their academic career, 
including on the STAR assessments: 

The class that they’re naming are a group of dyslexic children who are high 
achieving, bright kids and have had these accommodations to access the 
curriculum throughout their educational opportunity. They have an outstanding 
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attorney who is alleging it’s a violation of both federal law and IDEA and he has a 
good case. 

Our informants also discussed the waiver policy that CSBE established in 
response to the legal challenges against the CAHSEE and the uproar in the special 
education world. As this informant indicates, CSBE was initially prepared to deny 
diplomas to students with disabilities who used the banned accommodations, but had to 
backtrack: 

I think they were intending not to give diplomas to kids who they thought could 
have been taught to read or compute. But there was quite a furor over this in the 
special ed world. It made all the newspapers. 

It appears that CSBE finally listened to CDE staff and agreed to let students with 

disabilities take the CAHSEE with the previously disallowed accommodations if they 

needed them. If the student then achieved a passing score of 350 or above on the 

CAHSEE, he or she provided the CSBE with supporting evidence and petitioned for a 

waiver. Once granted, the waiver would enable the student to graduate with a regular 

diploma despite not obtaining a valid score on that portion of the CAHSEE: 

Basically this was what we encouraged them to do — was let them just take the 
darn test with these disallowed accommodations. They can’t actually pass the test, 
but if they can achieve a score that’s equivalent to a passing score , then they take 
that with some other information that the board wants as corroborating evidence 
that these kids are actually performing at a high school level, and they get a 
waiver from the passing requirement. 

Another infonnant explained that CSBE later planned to ask the legislature to 
allow local districts to grant waivers to students with disabilities, rather than keeping the 
decision at the state level. At the district level, our informants commented that there was 
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a lot of confusion and frustration with all the changes. Concerning the CAHSEE, one 
informant commented: 

So the latest issue was certainly over students with disabilities because the state 
board of education had ruled that any student using a calculator or any student 
having his or her language portions read to them would have an invalid score. 

Several infonnants in one district commented that they were trying to help parents and 
teachers to understand the CAHSEE process and to make appropriate accommodation 
decisions: 

Now I think we are engaged in helping parents and teachers understand, is this the 
correct accommodation that the student should be getting? It should not be just 
because they are going to be having a better chance if they have a calculator on 
the math portion and I would say that is our current challenge. 

As the following quotation illustrates, informants from the other district expressed 
similar thoughts. One infonnant reflected on the guidance from the state, commenting: 
“We’re given such really poor direction from the state and it comes at the last minute and 
people always get frustrated.” 

In this section, we have identified several issues that indicate a lack of attention or 
forethought to the needs of students in special education in California’s evolving 
accountability system. First, findings highlighted the tension between the initial 
assessment and accountability system and the participation of students with disabilities in 
state assessments. As the data indicate, it is likely that the scores of a significant number 
of students with disabilities were not included in the API system due to the choice of the 
SAT9 as the main assessment instrument. Perhaps the most prominent example of this 
lack of attention to the needs of students with disabilities was the implementation of the 
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CAHSEE, which brought the state into conflict with the leading federal disability rights 
statutes. We also found evidence that although the introduction of the CST meant that 
most scores from students with disabilities would be valid and thus included in the API 
system, schools may take action to protect themselves and their API. Our informants at 
CDE reported that schools may act to protect themselves by continuing to exempt 
students with disabilities from state assessments or allowing them to take out-of-level 
assessments. Evidence from the district level interviews conducted in June 2002 
suggested a mixed pattern of compliance with the state’s participation policy. However, it 
appeared that the motive behind exemptions was not to protect the district or its schools. 
Rather, exemptions were provided in cases where there was a belief that students in 
special education would be harmed by participation. 

In the next section we present findings on the extent to which the state and the 
local districts have the necessary resources, capacity, and intent to implement the next 
round of standards-based reform initiatives. In this section, informants will talk about 
their overall capacity to implement reforms as well as their capacity to implement 
reforms for students with disabilities in particular. 

Building Capacity 

The success of standards-based reform depends heavily on the ability and 
willingness of all levels of the public education system to respond in appropriate ways to 
the information provided by its accountability system. In this section, we address the 
issue of system level capacity across three related domains: (1) the state and the districts’ 
capacity to direct resources to support reform, (2) alignment with state standards, and (3) 
system support for students in special education. 
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State and Districts ’ Capacity and Willingness to Support Reform 

Financial resource limitations. The issue of how best to allocate resources to 
support reform was discussed at both the state and district level. At the time of our state 
and district interviews, these entities were experiencing significant budget shortfalls 
which impeded their ability to implement the PSAA. Over the preceding decade, despite 
declining revenue, the state had invested large amounts of money into developing the 
CL AS, curriculum and instructional materials, and professional development training that 
supported a constructivist view of education. 

Following the collapse of that initial reform effort, the state invested yet more 
money into education reform. The state developed new content standards and curriculum 
frameworks, held new textbook adoptions, redesigned teacher education standards, 
purchased the SAT9 and developed the CST, and provided funding to support the PSAA 
sanctions and rewards requirements and other initiatives including almost a billion dollars 
for class size reduction and early literacy (Carlos & Kirst, 1997). 

When we conducted the state level interviews in October and December 2001, the 
state’s financial situation had continued to worsen. This had occurred at the same time 
that high stakes accountability reform was beginning to be fully implemented. As one 
informant told us, “We can’t cut remediation for the CAHSEE, so my guess is that those 
are not likely to be affected by the budget because those are kind of core programs. 
Probably the trimming will take place in other kinds of programs and that kind of thing.” 
Another infonnant commented, “Our economy, it’s really awful and we are limited in the 
support we can give the locals.” Another infonnant explained that her division was 
responsible for providing information on the content standards, curriculum frameworks, 
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assessments, and textbooks and instructional materials to local districts, but the budget 
situation made these tasks very difficult. As the following quotation illustrates, CDE 
relied on the textbook publishers to a certain extent, along with the state Web site and 
mailings: 

You probably heard about the deficit we’re going through? I’ve already lost a 
couple of positions here and funding mid-year, but the lack of our own staff at the 
state level means we can’t do the outreach that we’d like to do to make districts 
more aware of the changes. I mean, granted we do the mailings, we have a great 
Web site, but we can’t be out in every district talking about the content standards 
and assessments. So one of the things California relies on is the awareness that 
publishers go between those various aspects and the expertise at the local level of 
their curriculum coordinators and others, we hope that they are really paying 
attention to the assessment components. 

An informant who was very knowledgeable in special education policy pointed 

out that although she had a large staff compared to other divisions in the CDE, her special 

education department was proportionally very small compared to those of other states: 

It’s incredibly limited. I have a huge staff in comparison to many of the other 
divisions here. But if you compare it percentage-wise we have probably the 
slimmest staff of any staff in the US. We literally have less people than states that 
are a quarter of our size. 

This infonnant also raised the issue of the CAHSEE and its potential financial 
impact on districts. Under IDEA, students with disabilities who do not graduate with a 
regular diploma are eligible for services through age 2 1 . The introduction of the 
CAHSEE as a requirement for graduation could create a large group of students who 
would be eligible for additional special education services and funding, which could be a 
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considerable burden for the school system. As the following quotation illustrates, no one 

had considered this when the CAHSEE was developed: 

Well, no one understood that there is a fiscal impact. A school district is the fiscal 
agent to be responsible for that kid’s education until they graduate with a high 
school diploma or until they’re 22. We’re looking at less than 40% of our special 
education population who pass this exam, then what will this do to school 
districts’ costs for alternate educational programs that last longer, for vocational 
training programs? 

The financial outlook was very similar in our two study districts. One informant 
described how student and teacher support programs were to be cut, and that the teaching 
force was being reduced: 

There are huge spending cuts. What’s going are our after school tutoring 
programs, our Saturday program. We had to cut three million dollars for the 
tutoring programs for kids. I would say that the other thing that has been cut — you 
have heard about ‘Teacher support.’ That is being cut next year. We are already 
holding staff reduction meetings to see where we can then send more teachers 
away. 

In our second study district the financial situation was exacerbated by decreasing 
enrolments as families moved away, and a decision to reduce the number of transfer 
students coming into the district. According to one informant, district administration and 
parents did not like students from other areas having access to such a good education. 

Our budget this year particularly is very challenging. Our enrollments dropped 
because people with families have moved away and our amount of funding 
coming into the district has dropped. We had a significant number of inter-district 
transfers over the years and we stopped those because we felt that the students 
within our own district should come first. Parents said, too, “We really want to 
have our children have access to the best, but we don’t want to necessarily have 
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the services diluted by having students who come from the outside.” 



According to another participant, the legislature’s requirement that districts ensure 
that each student has textbooks aligned to the state standards before instructional monies 
can be spent elsewhere was problematic. As the following quotation illustrates, books for 
other subjects were also needed, but the district did not have the money: 

So now with this budget crunch, it’s like, “where is the money going to come 
from?” We’re a bit concerned about how we buy Spanish language books, French 
books or auto repair books and so forth, and that’s by state initiative and, well it 
wasn’t the state department, it’s the legislature. They have not considered that. 

This infonnant also discussed an additional amount of money that the district 
would receive if it managed to get the new language arts textbooks in the classroom by 
September. As this informant told us, the district had little time to go through the 
decision-making process, but the money was much-needed: 

Plus, they just adopted Language Arts in January and they were saying if you 
have them in the classroom in September, then you’ll be able to get this extra pot 
of money, about $32 per student. That isn’t a huge turnaround time to look at 
materials, their publishers, convene a committee, have parents review, and 
everything and make a decision by June so that you can have materials in the 
classroom in September. If we don’t get that extra money, I don’t know how we’ll 
be able to buy everything. 

One of our informants explained that the state had given the district extra money 
about two years ago and the district had put most of this into teacher salaries. Now the 
district had to maintain the higher salaries, “Two years ago, when districts throughout the 
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state were given quite a little bit of money, in our district, all of the money went for raises 
for teachers’ salaries so we had to sustain that higher salary level.” Finally, another 
informant from this district described how the district was trying to keep the budget cuts 
out of the classroom by cutting central office staff, paraprofessionals, reducing the 
number of outside consultants, and limiting the number of field trips and extra curricular 
activities: 

The budget definitely has affected us. Here in this office we lost 2 1 people last 
year and so now this year there are more cuts here at this office because we tried 
to keep it away from the school setting. But there are some reductions there, 
mostly in paraprofessional, well, classified folks, and then also some reduction in 
the amount of field trips, the amount of co-curricular after school kinds of 
activities. I think we feel that at this point we’re still able to meet those goals and 
meet those needs. We’re pretty careful about how we spend our money. We have 
a lot of folks in our district who really have had a lot of training in their careers 
that can provide some of the leadership that we need. We don’t have to go to the 
outside. 

Human resource limitations. Like most states, California was experiencing a 

severe shortage of certified teachers, with special education teachers being particularly 

hard to recruit. One state level informant stated bluntly: 

It’s horrific! We need 300,000 teachers in the next five years; partly because of 
reduced class size, partly because of the pay — the economy, and partly because 
we have a huge number of folks my age who are getting ready to retire. 

When we visited our two districts in 2002, they each reported that they also faced 
teacher shortages, but perhaps to a lesser degree than the state. One informant explained 
that enrollment in the district was growing and the surrounding area provided other 
teaching opportunities for experienced teachers: 
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We definitely have a teacher shortage. We have been hiring about 500 new 
teachers a year for the past five years. We have been growing more than losing 
teachers really based on enrollment vs. retirement. It is really hitting us hard. 
Also, being located where we are, people have lots of options for working. So a 
lot of teachers start here, but then they go and work in Orange County schools or 
they go to smaller districts. They teach here to gain experience. 

This infonnant explained that the district did not have a problem finding 

replacement teachers, as they received a large number of applications each year. The 

challenge for the district was processing the applications efficiently: 

The flip side is we get about 7000 applications a year for those 500 positions. But 
the challenge is being able to really go through those in an effective and efficient 
manner and trying to pick up the cream of the top people we want. 

However, this informant also reported that it was difficult to attract qualified 

teachers and that around half of the applications came from individuals who were not 

qualified to teach. As the following quotation indicates, the percentage of new special 

education teachers without certification was higher than in regular education: 

Roughly 50% are not certified when they come to us. They are at various stages 
of not being certified, but not certified, nevertheless. In special education, of 
course, it tends to be a lot higher percentage of people. So teacher quality because 
of that can be little suspect and it’s hard to attract higher quality people to the 
district. 

The situation in our other district was very similar with one informant explaining 
that people found it very difficult to afford to live in the area on a teaching salary: 

It is really a concern and it grows and grows every day. But there seems to be 
different reasons why we are getting a teacher shortage now. We are finding that 
people can’t afford to stay in the Bay area. 
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The same informant commented that, due to the downturn in the economy, there 
was no shortage of people wanting to enter the profession. However, there remained a 
shortage of qualified teachers: “The economy is alleviating difficulties and shortages. I 
mean there are still shortages of certified people, but there are not shortages right now of 
people who want to teach.” 

As the following quotation illustrates, a year earlier this district had taken action 

to solve its special education teacher shortages: 

If you had asked me about special education teacher shortages last year I would 
have said, “Yes we have an acute shortage.” Last year we hired seventeen 
teachers on waivers, which means that they were not credentialed, but we needed 
them in the classroom. 

The informant went on to explain that the district had established a pre-intern 

system to encourage individuals to stay and become certified: 

This year we have been hiring very few because all of the people that I have on 
waiver from last year I am getting them in an interim pre-intern program as a 
condition of employment. What’s been missing is that we haven’t had this nice 
structure for the special education teachers to go into and last year I hired so many 
and we had no place for them to go and so they dropped out and so we lost them. 

Teacher certification issues. Teacher credentialing in California is not part of 
CDE and as a result few informants discussed this issue in much detail. One informant 
was critical of the power of the Commission on Teacher Credentialing, stating that the 
current system produced poor-quality teachers. According to this informant, the 
Commission needed to make changes at the higher education level and had not done so: 



53 




Nobody is at the table for teacher credentialing, except the Commission on 
Teacher Credentialing. They are a powerful entity unto themselves and they’ve 
mucked around with credentials and college requirements and we’re getting very 
poor-quality teachers. They’re very busy putting together the graduation 
requirements and the testing requirements for prospective teachers, but they’re not 
looking at turning the university system on its ear, which is what they have to do. 

Another infonnant at the state level stated that the larger school districts in 
particular tried to develop relationships with nearby universities and colleges to ensure a 
supply of new teachers, “A lot of school districts, particularly large school districts, build 
relationships with their neighboring universities and those teacher training institutions. 
They’re looking at creative ways to encourage people to come into teaching and to stay in 
teaching.” As the following quotations illustrate, when we visited our two districts in 
May and June of 2002, this view found strong support as both districts had cultivated 
relationships with nearby universities: 

We have a partnership program with our two California state schools and there 
are a couple of other programs that we are coordinated with. Primarily though we 
work with those two institutions and our special education program is with them 
too. 

An informant from our other district commented: 

We have a very close relationship with our Cal State school as far as being able to 
have student teachers placed here. We are six miles away from the university and 
they use our schools extensively for student teaching placements and most of our 
veteran teachers have gone through their program. 

Both districts had adopted a “grow your own” teacher certification program as an 
alternative route for teachers to become fully certified. However, the approaches adopted 
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were different in each district. In our larger district there were many more people who 
were teaching on emergency waivers and had been doing so for some time. One of our 
informants explained that the district was trying to track down uncertified teachers and 
get them into alternative certification programs. If a teacher did not agree then he or she 
was dismissed: 

Oh my gosh, 50% of the teaching force in our district are non-credentialed, but 
now because we have got the database, we are moving in on it and they have to 
show evidence that they are in the university program working on their credential. 
We have dismissed a couple of teachers this year because they have been non- 
credentialed for eight years, no coursework. 

In one of our districts, teachers enrolled in an alternative certification program 
were already working in the classroom as the teacher of record and attending classes in 
the evenings. As the following quotation illustrates, these teachers had full responsibility 
for the teaching and learning of the students in their classrooms: “They are working full- 
time in classrooms with complete responsibility for children and taking coursework and 
completing their fieldwork simultaneously. These people are pretty much all teachers of 
record.” 

In our other district, the alternative certification intern program was part-time, 

with the teacher interns gradually taking over responsibility for the classroom as they 

develop their knowledge and skills through coursework and observation. As the 

following quotation illustrates, the district adopted this “in-between” approach to avoid 

the teacher burnout that occurred when an intern was studying and teaching full-time: 

We like to stress the part-time as we feel that it is very challenging for somebody 
who has never been in the classroom before to take on a full-time internship. 

What we are hearing is that there are a lot of burnouts for these emergency permit 



55 




people. We wanted to alleviate that and it is our that philosophy that in order for 
someone to be successful in an intern program, they cannot be expected to be the 
teacher of record a hundred percent of the time, and so we build our programs so 
that they have part-time internships. 

This district also had a pre-intern program, designed primarily for second career 
individuals who did not have a degree in a content area: “We have applicants who are 
engineers or business people or psychology majors who have not completed a California 
pathway to becoming a teacher, and we don’t want to send them back to get another 
bachelor’s degree.” As the following quotation illustrates, the pre-intern program was 
designed to help prospective teacher interns pass the subject matter equivalency and 
pedagogy tests to get into the alternative certification program: 

So the pre-intern program is designed to help or get that very initial equivalency 
or subject matter that makes them eligible to even get into an alternative 
certification program. We provide the services to help them to do that and then 
transition them into the alternative route. 

Focus on State Standards 

When we conducted the state-level interviews in October and December 2001, 
several informants commented that it was very important for local school districts to 
align their instruction with the state content standards to ensure that the districts would do 
well on the state assessments: “These new tests, the Standards tests, are directly tied to 
the State standards and they are really driving curriculum and instruction. Clearly, any 
place where they are focusing on the State standards is going to do well.” Another 
informant explained: “The message is strong and clear to the teachers and to the field in 
general that the depth and breadth of the standards should be taught.” 
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Informants also commented that districts varied in their level of readiness for the 



new tests although, according to state-level informants, they had received plenty of 
advance notice: 

The schools and districts knew that it was coming. There are probably many 
places where they’d like to see all the standards-based tests coming in now and 
then there are other places that probably feel like, okay, we’re just sort of 
understanding what the Stanford 9 requirements are and trying to address those 
learning outcomes and now here comes this standards movement. 

Another infonnant echoed this viewpoint, pointing out that although the new 
standards were challenging the state had provided a lot of support over the previous 
years; 

I think we are asking a lot of our schools and districts, and at the same time it’s 
not beyond their reach. This hasn’t happened in a vacuum and there are so many 
systemic things that are going on. I don’t think there’s an excuse to say, “I just 
don’t know what to do.” We have nothing. 

Informants also discussed the textbook adoption process, pointing out that the 
most recent adoptions required publishers to include a special needs component to their 
standard materials. According to one informant, the CSBE and the Curriculum 
Commission did not want the needs of diverse populations to be treated separately or to 
be addressed by providing a watered down curriculum: 

Provision is not an afterthought and it’s not separate from the regular curriculum. 
That was not allowed in this adoption. This is really, really critical to the Board 
and the Curriculum Commission. Students are to be part of the entire curriculum 
through the day and then there’s additional support interfaced with the 
curriculum. It’s not, excuse the term, “dumbed-down,” it’s not simplistic, it’s not 
used in a way that differentiates in a negative way. 
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Finally, state-level informants commented that struggling districts and schools 
had to make some important decisions on where to focus their resources. Unsurprisingly, 
the state did not relish the idea of taking over schools. As the following quotation 
illustrates, the state was hopeful that the help it provided would turn districts around: 

So those schools are going to need to make the tough choices about where to 
focus their staff and instructional energy, and there’s a wide range of things 
available if a school is still struggling. But we all hope and expect that we’re not 
going to need to be getting involved and there will be a very small number of 
schools that fit that bill, and instead the schools will be improving. We all have a 
vested interest in that and I would be shocked if anyone says, “Oh, the state really 
wants to take over schools.” That’s the last thing anyone in their right mind would 
do. 

Another infonnant expressed more reservations about the impact of standards- 
based reform, especially on students with disabilities. As the following quotation 
illustrates, access to the full general education curriculum for this population was 
problematic at best, but without it students with disabilities would suffer: 

Well, I think if everybody is doing a good job of standards-based instruction, and 
that’s a huge caveat, then it is probably going to be okay. But everybody is not 
and that is where it is problematic. I think that special education students are not 
getting full access to the core because they don’t read well and they don’t do a lot 
of things well. So they are not getting the full access. Even though we say they 
are, they are really not. 

When we visited the local school districts in June of 2002, one of our study 
districts was very positive about their alignment with the state standards while the other 
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felt that they did not receive the help they needed. In the former district, informants 
explained that the district adopted their standards before the state did: “We came up with 
our standards first and they came up with theirs that really are ours.” 

Informants explained that the district had decided several years ago to focus on 
literacy: 

At a certain point in time we were going in too many directions so we decided to 
concentrate on literacy. Because if they can’t read then they can’t do much of 
anything else. That was a very strong focus and still is. 

As the following quotation illustrates, that district now focuses on literacy and 

mathematics, particularly in low perfonning schools: 

Now obviously we’re bringing in math and you have to do that. In some schools 
that are low performing we have to say, “OK. We’re going to do literacy and 
we’re going to do math,” and forget everything else, because they have to do this. 

As one informant indicated, new teachers in particular found it difficult to teach reading 

across the content areas and needed additional support from the district: 

The trick was teaching teachers how to teach reading through the content area. 
New teachers found it difficult at first as they can’t see the whole picture. So they 
don’t see how the pieces fall in place and you can’t really teach that. 

As the following quotation indicates, the district had adopted several reading 

programs, such as Open Court, Breaking the Code and SOAR to Success. In addition, the 

district had adopted Madeline Hunter’s Elements of Effective Instruction and had created 

literacy and mathematics coaches to support teachers: 

We are doing the Elements of Effective Instruction. That’s one of the focuses that 
we have in the district, along with Open Court and Breaking the Code, and SOAR 
to Success. We have literacy and math coaches for teachers,, and they work 
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closely with the principal, and then we also have coaches for the principals. These 
coaches come in and provide assistance to the teachers. 

In terms of alignment, informants stated that the district had implemented 
curriculum mapping for the schools. As the following quotation illustrates, at one time 
schools varied in terms of grade-level continuity. This infonnant believed that the 
continuity of curriculum served to build in grade level accountability, as peer pressure 
forced teachers to teach the curriculum: 

In a lot of the schools there was no curriculum mapping. So, there was no 
continuity at the grade level. What it does is, it builds in the accountability at the 
grade level and then peer pressure begins to take over and they support 
themselves. They meet on a regular basis at the grade level. They have some of 
the district coaches come in and provide support. So, they are just not out there 
floundering. 

Informants indicated that professional development also was focused on specific 
areas. However, this practice was mandated by the CDE, which applied strict rules 
concerning what activities districts could spend their monies on: “We mandate that the 
training be about the frameworks and the instructional materials. So those can’t be 
ignored. You can’t have training any longer if you want these funds.” However, 
according to several informants, this focus was a positive move: 

Our staff-development is very focused now, it’s not scattered all over the map. 
What we have is an ongoing focused staff development for teachers and principals 
and all the staff. So, it really has made a big difference in what’s happening in the 
classrooms. 

On the other hand, informants told us that the state had reduced the number of 
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professional days and this made it difficult to provide all the training that teachers 
needed. A state informant also addressed this issue, commenting: “This governor, one of 
the first things he did was do away with some of the staff development days. So we have 
less time and less money.” As the following quotation from a district level informant 
indicates, the reduction in professional days had an impact: 

Well actually, a couple of years ago the state cut back on the number of school 
improvement days that teachers take for professional development. So now 
professional development is whatever the school can do during collaboration time 
which is also going — won’t have next year — and the three state-allowed ones. So 
it’s troubling. We want teachers to know about the programs and so forth. 

Finally, though, as the following quotation indicates, not everyone adopted 
standards-based reform willingly. However, as this informant pointed out, the district had 
uniformly high expectations and if individuals wanted to stay in the district they simply 
had to accept that: 

I think it’s mind-boggling that you still have the reluctant people. I mean, you 
know that you don’t get rid of all of them but it puts a lot of pressure on them in 
schools. I heard people say this is just “too much work,” and “I just can’t do it.” 
And I think, well, where are you going to go in this district where that won’t be 
the case? 

In our second district, informants commented that the district had yet to really 
understand what was going in terms of assessments and standards. As the following 
quotation illustrates, there was frustration and anger about the frequent changes and lack 
of guidance: 
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That’s part of the problem because it all changes you know. So this year, of what I 
know, it’s the SAT9, and some California Standards test. But then none of our 
teachers have ever given it or been part of it. So we are supposed to know how to 
explain it to the parents when it’s a brand new test, a whole new animal! I’ll tell 
you it’s always the same and this makes our teachers angry — “What do we do? 
Nobody is telling us what to do. How do we talk to parents about the test? We 
have never even seen the test.” 

The district had made some changes to improve curriculum and instruction. For 

example, one informant reported that the district had brought back curriculum leaders in 

four core content areas. However, due to cuts, those positions were disappearing: 

Years ago, we had a structure that was a curriculum leader in each of the four 
main subject areas of each school. Then our previous superintendent dropped that 
position for about four years and then, two years ago, we brought the curriculum 
leaders position back and that has been very helpful because they meet with me 
and we meet together by curriculum area. We hear the questions that teachers have 
about implementation, we are able to address those, and then it goes back to the 
schools. Now, unfortunately, we’re looking at having to drop those positions for 
next year. 

One crucial role performed by the curriculum leaders was to vertically map the 

district’s curriculum against the exam blueprints. As the following quotation indicates, 

this process went from the CAHSEE right back to Kindergarten: 

What we did when the exit exam blueprints were printed, we took those standards 
and had a curriculum leader go to the different grade levels in math and language 
arts and highlight if a child has to be able to do this for the high school exit exam, 
where’s the thread back to kindergarten, 1st grade, 2nd grade, and so on. The 
teachers were able to identify those key standards so that children would be able 
to be prepared and that teachers knew what they needed to teach. 
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In terms of instructional and content area programs, the district adopted the same 
programs in all of its schools. However, as the following quotation indicates, the district 
was beginning to consider the needs of individual schools while still using district-wide 
basic programs: 

The schools never went off on their own like a lot of places, where each school is 
an entity and they do what they want and they pick their own materials and do 
their own thing. We never did that. It was always a district. We’re beginning to 
implement a little bit differently based on the needs of the schools, but still it’s the 
same basic program that everybody sort of signs on to. 



System Support for Students in Special Education 

When we visited the state in October and December 2001, NCLB had not yet 
been implemented. Nonetheless, the issue of comparable subgroup performance was at 
the forefront of informants minds, 

I think any accountability system that’s not talking about those issues is probably 
missing something and is probably doing something injurious to one set of kids or 
one set of schools or other because it is a balancing act between the political and 
the policy and technical. 

As the following quotation illustrates, there was some debate over the value of 
adding students with disabilities and students who were limited English proficient to the 
list of subgroups that schools were accountable for in the state API system: 

We have seven ethnic groups and socio-economically disadvantaged. If you did 
have English learners and students with disabilities as separate subgroups, there 
was the issue that you got a category that is heavily correlated with another 
category that you’re already counting. More importantly, though, establishing 
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additional subgroups is effectively another hurdle for these award programs for 
meeting growth targets. The requirement of comparable improvements would 
actually serve to disadvantage some of the various schools where you are seeking 
the help. 

The same informant added that reporting subgroup performance, but not making 

improvement a requirement for schools, could be a good compromise. However, this 

informant also mused that this practice could encourage people to blame certain 

subgroups for having a negative impact on the school’s or district’s reputation: 

The compromise solution to this, of course, is to report it out without making it 
binding, and that solution has always intrigued me. The downside is we certainly 
want to steer away from anything that’s getting into a blame game. There’s a fine 
line between providing the information, but with these stakes, we need to be very 
thoughtful about how we do that. 

However a different informant at the state level had the opposite opinion, 

commenting that reporting information about students with disabilities focused attention 

on them and also allowed schools with problems to see that other systems were more 

effective. As the following quotation illustrates, superintendents in particular were not 

always happy about having their district’s performance out into the open: 

Last year, we published for the first time every district in California that has more 
than 200 students with disabilities. We had a few excited superintendents! 

They’re paying attention now because it shows their district’s data plus the state 
comparison, plus their comparable districts. Which is the other interesting piece. 
We’re showing them, if they have lousy dropout rates, there’s a comparable 
district that has a great track record and they’re starting to talk to each other. Just 
publishing stuff changes how people do their jobs. 
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Other informants expressed the opinion that accountability highlighted the poor 
performance of some students: 

What I have observed just in the last few months is that the shining the light 
aspect of an accountability system includes your vulnerable populations or at-risk 
and your special needs populations in a different light than you’ve seen in the 
past. 

Several infonnants commented that they could no longer ignore the needs of 

students: “Educational outcomes for certain populations were so bad that it wasn’t 

possible to ignore them any longer.” Another infonnant stated, 

I think that educators and policy makers, especially at the state level, have just 
gotten to a threshold where it’s no longer acceptable. I think that the numbers 
have just been so dismal that there was just no way you could do anything but act. 

In discussing outcomes for students with disabilities, one infonnant said bluntly: 
“As a professional in special education, if you look at the results that we’ve had for our 
children, it’s abysmal.” 

When we visited the districts in June 2002, more information was known about 
NCLB requirements. In one of our districts, informants were very supportive of having 
students with disabilities as a subgroup for AYP purposes, “The best thing that can 
happen for us is that special education is a subgroup. I want them in. I think it is a great 
idea.” As the following quotation illustrates, by making students with disabilities a 
subgroup, district personnel hoped to hold principals and schools accountable for their 
progress and not allow them to get away with blaming the special education teacher: “I 
don’t want to hear that our special ed teachers are shitty teachers. I want to know the data 
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that shows whether or not principals and other administrators are holding themselves and 
their staff accountable.” 

Several other informants stressed the importance of using data to determine 

program effectiveness for regular education. As the following quotation reveals, the 

practice of using special education performance data for the same purpose was slower to 

take off and met some resistance, but was beginning to be effective: 

This district is totally data driven. Everything is data driven. It is all there in 
general education and now we are saying it is no different for special education. 
For the first time, we took all the data for special education kids and 
disaggregated it and gave it to each principal. I said to them, “These kids are 
failing because they do not know how to read.” Squawk, squawk from the 
principals. “What have you done lately for the kids that are failing?” and they do 
not know what to say. I mean, if I give you a list of all your resource room 
students and all of them have two or more Fs and ask what have you done? They 
have no defense. 

The situation in our other district was rather different. The district clearly 
regarded itself as high-performing and had an API of 712 in 2002, which increased to 734 
in 2003. According to informants, the administration, including the Superintendent, was 
focused on high expectations: “Our superintendent is one who believes that all kids can 
achieve and that’s what we’re doing. It’s pretty much a part of our culture here.” 
However, several informants reported that the district had challenges with certain groups 
of students who did not come to school ready to learn: “But it doesn’t mean that we don’t 
have some significant challenges in helping kids who perhaps don’t come to school with 
the same, I mean they don ’t come to school with the same level of preparation or 
experiences.” Another informant explained bluntly that some students, particularly those 
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who were not ready to learn, just fell through the cracks in the district. Interestingly, this 

informant felt that too much money was being spent on students with autism, intimating 

that the needs of other students with disabilities were not met: 

Now I’ve got a lot of different perspectives to others in the district, but the kids 
who don’t have that kind of support from the home and that kind of thing, I think 
they’re just going to fall by the wayside, despite some real good efforts. And part 
of the thing is also, when you’re talking about giving services to kids with 
disabilities, there’s so much money going to autism. We have model autism 
programs and they are really good, but they are high cost programs. 

In this district, there was evidence to suggest that the academic performance of 

students with disabilities was not a focus for the administration. This is not to say that the 

district was inattentive to the needs of this population. They were very proud of the newly 

established program for students with autism, and reported high levels of parent 

satisfaction with the special education services. As the following quotation indicates, the 

district was preoccupied with needs of its larger English learner population and did not 

pay much attention to data on the performance of students with disabilities: 

You know, we haven’t really done a longitudinal look at special education kids. 
I’m not saying it’s something we shouldn’t have been doing, but in terms of how 
priorities are worked out the English learners are kind of where the focus has 
seemed to need to be, much more so than in special ed. So, I wish I could say we 
followed it carefully. . .we have noted it but we really haven’t. . .it’s not something 
that we’ve been really focused on, looking at with our teachers. 

As part of our document analysis we had collected performance data on students 
with disabilities in all study sites. When we verified performance data from this district, 
informants expressed surprised at how poorly students with disabilities scored on the 
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latest STAR assessments: 



You know, I haven’t been able to really figure that out. Our special education 
scores are lower than the state average special education scores, whereas, in 
general, our scores are a little bit higher. I mean, we generally tend to be five or 
ten points higher than the state average. I discussed this very briefly with the 
director of special education and we’re not really sure what the issue is and we 
don’t really know why. We were kind of surprised that they were lower. We 
haven’t done a lot of our own analysis of those tests, so we really can’t say. 

There was also evidence suggesting that this district was just beginning to look at 

its data for other groups, and to use data to make programmatic changes. As the 

following quotation indicates, teachers were not skilled in data-based decision-making, 

and the district needed outside help to bring about what amounted to a paradigm shift: 

Two consultants have come to work with us for two days now and they really talk 
about data. The other day one was here and she was looking at our consolidated 
plan and she said, “Well, you guys have nice goals but you have an achievement 
gap and you are not addressing that. You need to look at where your children are 
scoring and you need to figure out how you are going to get the groups that aren’t 
scoring so high up with the others.” We are just developing those this year and 
everything has to be tied to that data — we are being forced into it. But to get your 
teachers talking more about data and the actual use of it right there in the 
classroom on a daily basis, we aren’t there yet. 

In this section we have addressed the need for alignment of the state’s content 
standards, assessments, and accountability system. At the state level, belief in the logic of 
accountability theory is very evident. School systems are aligning their curriculum in 
order to do well on assessments and thus avoid the negative consequences of poor 
performance. However, at the state and the district level, the problem of capacity loomed 
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large as each level experienced teacher shortages and dwindling resources of money and 
time. One district appeared to be much further along in tenns of aligmnent and system 
capacity, while the other was slowly and possibly reluctantly responding to the state-level 
changes. 

The level of support for including students with disabilities in the accountability 
system varied considerably, especially at the district level. At the state level, there was 
evidence that support was not very strong for adding students with disabilities to the list 
of significant subgroups. Ultimately however, the CSBE added students with disabilities 
and LEP students to the API in order to align with Federal AYP requirements. In the 
following section we will present data on our third and final theme: the struggle for 
control of the education system. 

Competition for Control of Education Reform 

It would indeed be naive to ignore the role of politics in education refonn. As 
discussed earlier in Section II, education reform in California has at times been 
dominated by political wrangling, with the control of education moving from one group 
of stakeholders to another. When we visited the state in 2001 and the districts in 2002, it 
was clear that the struggle for control over education reform was ongoing. However, as 
pointed out in earlier sections, at the state level we only interviewed informants from 
CDE and did not obtain data from other stakeholders, such as the CSBE or other 
Commissions involved with education. The struggle over the control of education 
emerged in three domains or levels: between local districts and the state, between 
different stakeholders at the state level, and between the state and federal government. 
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Local Districts and the State 



One of the major tenets of standards-based reform is that all students, regardless 
of where they live in a state, should have an equal opportunity to learn content that the 
state deems important and valued. By its very nature, therefore, control of education 
gravitates from local districts toward the state level as the state decides what the content 
standards are, what the assessments are, and how schools and districts will be held 
accountable. In theory, the benefits of state central control and policymaking are 
consistent standards, clear guidance for instruction, strong consensus on goals and 
objectives, and greater equality in resource allocation and student achievement (Ball & 
Cohen, 1996; Timar, 2003). 

According to several researchers the California state legislature has become 
increasingly involved in matters of curriculum policy, and school officials and teachers 
have had to deal with an outpouring of legislative initiatives. For example, one informant 
commented: “I call it trickle down legislation. It’s like ‘hey legislate and it trickles down; 
and we figure out how to pick up the pieces and make it work.’ But there’s no trickle. 
Basically it’s damage control.” As part of the 1999 PSAA, the move from local to state 
control continued as the state legislature created new curriculum frameworks, adopted 
new textbooks and instructional materials aligned with these frameworks, and controlled 
how districts spent their professional development money. 

Although infonnants from CDE maintained that the education system was 
“locally governed” and that the curriculum standards were “voluntary”, control over what 
was taught was migrating to the state by way of control over textbooks, instructional 
materials, and money. One of our districts was particularly frustrated by this, with 



70 




informants pointing out that their choices were very limited: “When you only have two, 
it’s just very difficult because you just don’t have much of a choice.” Another infonnant 
discussed a specific mathematics program that the district had used in the past, but was 
now not on the approved list: “We have a lot of people who were very devoted to Dale 
Seymour, but we don’t have it anymore because it’s no longer state adopted. Our teachers 
had a really hard time making that transition.” 

Informants also discussed the increasing state control over money for instructional 
materials. Our informants told us that in the past K-8 had to use 70% of their instructional 
materials money for state-adopted materials, and a certain percentage could go for 
materials that had met legal compliance but were not adopted for the whole state. Then 
5% could be for any instructional materials that they wanted. However, as the following 
quotation illustrates, the CSBE Board had placed further restrictions on district 
discretion: 

Now, though, what they’ve said is that any of this new money that comes into the 
state can only be used for the four curriculum areas and you cannot buy anything 
else until you can certify that every child has a standard state program; book, 
materials, in their classroom in those four curriculum areas. So we’re a bit 
concerned about how we buy Spanish language books, French books or auto 
repair books and so forth. It’s the legislature and they have not considered that. 

Tension at the Intra-state Level 

At the state level, responsibility for the California public education system is 
dispersed across a variety of programs, agencies, and commissions that do not necessarily 
work well together. There was considerable evidence from informants at CDE of 
continuing tension between CDE and CSBE, or at least among certain dominant 
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individuals. According to some informants at CDE, the heart of the issue was CSBE’s 
inflexible and dogmatic approach to education reform, and the impact of this approach on 
diverse learners such as students with disabilities. As one participant explained, there 
were few disagreements over the goals of education reform, but some had concerns over 
the means adopted: “I think everyone agrees that for all students we want higher, better 
instruction and higher levels of achievement for all students. No one argues with that 
point. The issue is how we get there.” Tensions emerged in two areas: first, the initial 
design of the accountability system; and second, the educational philosophy behind the 
current reforms. 

The following quotation from an informant who had over 20 years experience in 

CDE exemplifies the views expressed by most of our informants from CDE: 

There’s been an attempt to implement a system where one size fits all. What the 
voices inside the department, and also the parents and people in the field who 
represent Title I students, or special education students, or English Language 
Learners, just to name three of them — and many, many others as well have all 
pointed out is the inflexibility of the initial notion, the naivete of the initial plan 
for a standards-based system, and therefore, the need for accommodations in the 
system to serve the complexity of what children are actually like. 

Several of our state-level informants reflected that this inflexibility and 
prescriptive approach were responsible for the difficulties generated for students with 
disabilities by the CAHSEE. As the following quotation illustrates, the approach adopted 
toward this population as it related to accommodations and high school diplomas was 
unfair: 

This is the basis of the difficulties we find ourselves in with high school exit 
exams in particular. What it comes down to, is that fair or the right thing to do? 
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Let’s say a student enters into a special ed program in the 2nd grade and they sit 
in the IEP team meetings, and they set their goals and achieve these goals, and 
move right along and we say “you keep doing that and when you get to the end, 
you’re going to receive your diploma.” Well, two thirds of the way through their 
education all of a sudden we say- “Oh, well we really didn’t mean that. We’re 
sorry that you haven’t been involved in the right learning because now we have 
the high school exit exam and here it is. Good luck on that. 

Several of our informants reflected that conflicts over curriculum and instruction 

were a long-tenn feature of refonn in California, pointing out that the original battles of 

the 1980s had essentially resurfaced as standards-based reform: 

Remember that when our standards were created in California, they were highly 
controversial, and they represented the victory of one educational, political pole 
over those who lost out. So there isn’t an educational community embracing the 
standards. Those battles, which were originally in California in the early 1980s 
over curriculum frameworks and instructional methodology, continue to be almost 
the central feature here. The standards turn out not to be a consensus document, 
but rather an agenda document — an agenda of a subset of educationally concerned 
people. 

This infonnant explained that the direction and substance of education reform — 

what was taught and how it was to be taught — was under the control of a small group of 

individuals who shared the winning philosophy: 

We’ve got the standards, we’ve got assessments aligned to the standards, we have 
textbooks, which are now polar — not broad — but which represent the pole of an 
ideology with certain prescribed instructional strategies. So when the Commission 
on Teacher Credentialing says, ‘teach to the standards,’ they don’t mean teach to 
the standards they mean, ‘teach that instructional strategy.’ 
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A different informant described the impact that this approach had on teachers and 

their ability to teach diverse learners. As the following quotation shows, at a time when 

the student population in California was becoming more diverse, the pedagogical skills of 

its teachers were becoming more narrow: 

Our instructional materials adoption heavily focused on scripted programs. So 
teachers do not have the flexibility to design their unit of instruction and to be 
flexible in their own creative teaching methods. In fact, they’re scripted moment- 
to-moment. We’ve have 300,000 California teachers who need very intensive 
staff development to serve a diverse group of students in what, I would suggest, 
need to be flexible ways, yet we don’t do that. 

Other comments provided insight into the role that educators played in 
California’s reforms. As the following quotation indicates, teachers were seen as the 
problem and therefore could not be included in the search for a solution. This informant 
concludes by pointing out that the solution was therefore not an educational one, but a 
political one: 

The standards commission that developed the standards in California had very 
few teachers on it because teachers were seen as the problem and therefore not 
able to provide the solution. Teachers have low expectations for students and that 
was part of the problem. Therefore, they couldn’t be counted on to establish the 
higher expectations that we needed and so on. It’s always tricky constructing an 
educational system in a state that excludes teachers and I’d like to suggest that it 
is almost not an educational solution but a political one. 

Unsurprisingly, most of our informants at CDE were cautious in their predictions 
of the success of these reform efforts: 

It’s not at all clear that this is leading to nirvana. I mean it isn’t at all clear 
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that this is just what we needed. It’s wonderful to have the coherence, but 
it’s got to be coherence around meeting the flexible needs of teachers and 
students and we are not there. 



Another infonnant reflected that the theory was that once students had access to 
strong instructional programs their performance would improve. However, if this proved 
to be erroneous, then dropout rates could increase and students would not get diplomas: 

Well we may discover that it’s very difficult to teach them up to these standards 
and it would create an interesting situation, and that’s one of the things that our 
independent evaluation in high school exit is supposed to study. The intended 
consequence is that wonderful programs will be offered to them and they’ll reach 
the bar. What if the wonderful program is offered to them and they don’t reach the 
bar or they drop out? That’s an unintended consequence; there’s not supposed to a 
large number of students there. 

Comments from CDE infonnants also provided insight into how they believed 
CSBE viewed students with disabilities, especially regarding the CAHSEE. Several 
informants reported that dominant members of the CSBE did not believe that most 
students with disabilities, especially those who were learning disabled, were really 
disabled. The following quotations are reflective of many comments we heard: 

“One of the reasons that I heard for the CAHSEE policy was, ‘Well, dyslexic kids 
can be taught to read,”’ and, “I don’t think they’re being pushed hard enough now.” 
“[Name removed] will parrot children are in special ed because they haven’t been taught 
to read.” “[Name removed], she’s the one that’s visited so many places and districts to 
see kids that she knew were not learning disabled and not special education, but had not 
had the opportunity to learn to read.” 
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Another infonnant explained that before the CAHSEE could be made a 

requirement for graduation, the issue of opportunity to learn had to be well-documented 

because of the high stakes at the student level of not graduating with a diploma. This 

informant explained that the situation was more complex than many realized, because 

while students may be given multiple opportunities to take the exam, and be provided 

with remediation, some students did not have the basic foundations in the content areas: 

We don’t think we’ll be able to give the test for various reasons if a student hasn’t 
had the opportunity to learn the material on the test at least once. The problem is 
this weird situation where over a two or three-year period you’ve given a student 
two opportunities to learn everything on the test, but he or she didn’t have the 
precursors necessary to succeed on that. 

Tension between the State and USDE 

When we conducted the state-level interviews, there was evidence that the 
relationship between the state and USDE was strained, especially over compliance with 
the Elementary and Secondary Education Act (ESEA). We visited in Fall 2001 and the 
state assessment system, required by the Improving America’s Schools Act (IAS A) 

(1994) was yet to be approved by USDE because the state did not have all the elements in 
place: 

I think our next crucial step is to get the assessment system approved. See, it was 
supposed to be finished in Fall of 2001. We didn’t make that deadline. It’s not 
approved by USDE because we don’t have the math performance standards in 
place, our alternate system for special education is not done, and our alternative 
system for small schools won’t be ready until 2002. 

The same informant told us that the state wanted USDE to grant them a timeline waiver 
rather than requiring the state to enter into a compliance agreement: 
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I mean, we’re trying to convince the Feds that we don’t need a compliance 
review. What we need is time. The plan is there, it’s just that it’s not finished, and 
there’s so many political hoops you have to jump through to get anything done. 

In this section, we discussed how political tension affected the implementation of 
accountability reform in California. Tension emerged at three levels in the education 
system, but was most obvious at the state level between CDE on the one hand, and CSBE 
and the state legislature on the other. At the heart of the issue was a difference in 
educational philosophy between some informants at CDE, particularly those who had 
been there during the 1990s, and the newly elected members of the state board of 
education. According to our infonnants, the approach of CSBE and the legislature was 
too inflexible to easily allow for the inclusion of students who were different. Although 
the issue came to a head over students with disabilities and the CAHSEE, the needs of 
other students, such as English learners, andTitle I students, were not met by the initial 
accountability design. 

V. CALIFORNIA CASE STUDY DISCUSSION AND CONCLUSION 
In the previous section, we presented qualitative data, collected over a two-year 
period-from two levels of the public education systems, on the impact of accountability 
reform on students with disabilities in California. State-level interviews took place in 
October and December 2001, district interviews took place in June of 2002. The 
sequential nature of data collection enabled us to study the impact of state accountability 
reform on students with disabilities and to follow the concerns identified by state-level 
policymakers through to the districts. However, collecting data in this way meant that we 
were unable to obtain the initial views of state-level personnel on the impact of NCLB, 
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which was signed into law after the completion of the state-level interviews, or to study 
the further development and refinement of federal accountability policy as it related to 
this population. When we visited the districts in the following year, there was also little 
discussion of the impact of NCLB legislation. 

EPRRI staff interviewed 20 individuals, 10 at the state level and 10 at the district 
level. Findings are arranged into three broad topical areas: (1) fitting students with IEPs 
into the turbulent policy environment, (2) building system capacity, and (3) shifting of 
the locus of control over education. When considering the impact of standards-based 
accountability reform on students with disabilities in California, it is important to 
remember that the state had experienced a series of failed reform efforts. The current 
wave of reform continued and accelerated central control of curriculum and pedagogy, 
but did not vest it in the hands of education professionals. In addition, accountability for 
student learning was a recent concept for schools and districts. With this in mind, the 
following key findings emerged from this study. 

First, when we visited the state in October 2001, we found an accountability 
system that can best be described as a work in progress, with some of those involved in 
its construction being highly critical of the fundamental design. Informants from CDE 
spoke of trying to keep up with the many changes that had to be phased in, and of trying 
to maintain public confidence in the system. Infonnants from one of our districts were 
particularly critical of the level of guidance provided by CDE to teachers about the CSTs. 

Second, students with disabilities were not the focus of the reform effort and 
decisions were made that left students with disabilities outside the initial system 
established by PSAA. There is no evidence to suggest that these students were 
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deliberately and intentionally excluded; indeed CDE informants emphasized that they 
wanted the participation of as many students with disabilities as possible, but decisions 
by the state legislature and the CSBE suggest a lack of forethought regarding students 
with disabilities. For example, the choice of the SAT9 was problematic, as this 
measurement tool limited the practical inclusion of students with disabilities in the 
accountability index. Further, the key decision about accommodations and the CAHSEE 
embroiled the state in a high profile lawsuit that proved a considerable distraction for 
CDE, and revealed the presence of underlying tensions within the state. 

Although the introduction of the CSTs meant that more scores from students with 
disabilities could be included in the API, CDE informants feared that schools and districts 
would soon try to limit the impact of their scores on the schools rating, and would do this 
by exempting students with disabilities from the assessment or by giving them out-of- 
level assessments. Albeit unprofessional, these types of behaviors are unsurprising given 
the high-stakes nature of current accountability reform. 

In addition, we found that infonnants at both the state level and district level were 
divided over whether students with disabilities ought to be recognized as a significant 
subgroup, along with other at-risk groups. At the state level, some informants were 
concerned that students with disabilities could become scapegoats for a school’s failure to 
meet its API, while others pointed out that requiring schools to be accountable would 
force schools and districts to pay attention to test scores and better address the needs of 
this population. 

In one of our districts, students with disabilities were clearly included in the 
overall commitment to standards-based reform, and there was some evidence that 
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performance was improving. In the other district, however, the academic needs and 
performance of this population were not a major focus or concern. In this district, several 
informants retained the traditional view that it was unfair to expect students with 
disabilities to perform well on academic assessments. 

A third finding is that any refonn effort requires system capacity to support 
effective implementation of policy. State-level informants did not have the staff or 
resources necessary to support the local school districts in aligning their curricula with 
the state’s learning standards, and relied on textbook publishers to help out. Informants 
perceived this lack of alignment as a serious obstacle to education reform in some 
districts. One of our two participating districts reported that they had adopted standards- 
based reform before the state did, and had simply tweaked their existing curricula. 
However, our other district was still working on alignment and use of data to guide 
instructional decision-making. Infonnants at the state and district level spoke of the need 
to focus their resources and attention toward literacy and mathematics. 

The shortage of teachers was also an issue at the state level and the district level. 
At the state level, CDE informants admitted the extent of the shortage, but reflected that 
they had little control or say in developing a solution, which was the purview of the 
credentialing commission and the CSBE. Both districts had implemented alternative 
certification programs, although they differed in structure from each other. 

Our fourth finding relates to the tension among and within the various levels of 
the public education system. At the local level, one of our districts in particular expressed 
frustration over the lack of guidance from the state and the control exercised by the state 
over the choice of textbooks, instructional materials, and the way that professional 
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development money could be spent. Although the goal was to ensure alignment at the 
local level with the state’s standards, the means — controlling the purse strings — created a 
degree of resentment. Tensions between CDE and CSBE were very apparent, indicating 
that commitment to accountability reform in its current incarnation is by no means 
certain. Although CDE infonnants emphasized their commitment to standards-based 
reform, the means employed by the legislature and CSBE did not have the unconditional 
support of most of the CDE staff with whom we spoke. Comments from CDE informants 
also indicated feelings of impotency as CSBE, the state legislature, and various 
commissions made decisions that CDE perceived to be mistakes. 
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